Hugging Face's logo

abletobetable
/
rut5-base-absum-tech-support-calls

Text2Text Generation
Transformers
PyTorch
TensorBoard
t5
Generated from Trainer
text-generation-inference
Inference Endpoints
Model card Files Metrics Community
1
abletobetable commited on
Commit
b25a0d0
•
1 Parent(s): 6a2253d

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +29 -61
README.md CHANGED
@@ -14,12 +14,12 @@ should probably proofread and complete it, then remove this comment. -->
14
 
15
  This model is a fine-tuned version of [cointegrated/rut5-base-absum](https://huggingface.co/cointegrated/rut5-base-absum) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
- - Loss: 2.8739
18
- - Rouge-1: 0.4059
19
- - Rouge-2: 0.2831
20
- - Rouge-l: 0.3902
21
- - Gen Len: 15.5
22
- - Avg Rouge F: 0.3598
23
 
24
  ## Model description
25
 
@@ -45,64 +45,32 @@ The following hyperparameters were used during training:
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
  - lr_scheduler_warmup_steps: 200
48
- - num_epochs: 250
49
 
50
  ### Training results
51
 
52
- | Training Loss | Epoch | Step | Validation Loss | Rouge-1 | Rouge-2 | Rouge-l | Gen Len | Avg Rouge F |
53
- |:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:-------:|:-----------:|
54
- | 2.7022 | 2.78 | 50 | 2.2970 | 0.0 | 0.0 | 0.0 | 6.875 | 0.0 |
55
- | 2.2932 | 5.56 | 100 | 1.8183 | 0.0 | 0.0 | 0.0 | 10.375 | 0.0 |
56
- | 1.8234 | 8.33 | 150 | 1.4890 | 0.3588 | 0.2205 | 0.3262 | 14.0 | 0.3018 |
57
- | 1.3727 | 11.11 | 200 | 1.3740 | 0.3493 | 0.1653 | 0.3167 | 12.375 | 0.2771 |
58
- | 1.0367 | 13.89 | 250 | 1.3833 | 0.2607 | 0.0984 | 0.2331 | 15.375 | 0.1974 |
59
- | 0.841 | 16.67 | 300 | 1.3516 | 0.3713 | 0.1857 | 0.3594 | 16.0 | 0.3055 |
60
- | 0.7182 | 19.44 | 350 | 1.3607 | 0.3352 | 0.143 | 0.3233 | 16.125 | 0.2672 |
61
- | 0.5102 | 22.22 | 400 | 1.3673 | 0.36 | 0.1597 | 0.3349 | 16.625 | 0.2849 |
62
- | 0.4595 | 25.0 | 450 | 1.3715 | 0.3892 | 0.2153 | 0.3641 | 17.125 | 0.3228 |
63
- | 0.3886 | 27.78 | 500 | 1.4634 | 0.3801 | 0.2274 | 0.3682 | 16.375 | 0.3252 |
64
- | 0.3158 | 30.56 | 550 | 1.5124 | 0.3938 | 0.2319 | 0.3672 | 16.75 | 0.331 |
65
- | 0.2687 | 33.33 | 600 | 1.5868 | 0.3987 | 0.2568 | 0.3848 | 16.5 | 0.3468 |
66
- | 0.2361 | 36.11 | 650 | 1.6460 | 0.375 | 0.2107 | 0.3631 | 17.75 | 0.3163 |
67
- | 0.1991 | 38.89 | 700 | 1.6947 | 0.3605 | 0.2177 | 0.3474 | 16.25 | 0.3085 |
68
- | 0.151 | 41.67 | 750 | 1.8248 | 0.3832 | 0.2274 | 0.3559 | 16.5 | 0.3222 |
69
- | 0.1517 | 44.44 | 800 | 1.7884 | 0.4309 | 0.294 | 0.4184 | 16.875 | 0.3811 |
70
- | 0.1444 | 47.22 | 850 | 1.8519 | 0.3843 | 0.2107 | 0.3711 | 17.125 | 0.322 |
71
- | 0.1106 | 50.0 | 900 | 1.9637 | 0.383 | 0.2107 | 0.3691 | 17.5 | 0.3209 |
72
- | 0.0961 | 52.78 | 950 | 2.0718 | 0.3645 | 0.2177 | 0.3488 | 16.75 | 0.3103 |
73
- | 0.1131 | 55.56 | 1000 | 1.9935 | 0.3602 | 0.2153 | 0.3446 | 16.75 | 0.3067 |
74
- | 0.0996 | 58.33 | 1050 | 2.0616 | 0.4153 | 0.2986 | 0.3996 | 16.0 | 0.3712 |
75
- | 0.0663 | 61.11 | 1100 | 2.1466 | 0.4257 | 0.301 | 0.409 | 14.625 | 0.3786 |
76
- | 0.0789 | 63.89 | 1150 | 2.1657 | 0.4166 | 0.301 | 0.4009 | 16.0 | 0.3728 |
77
- | 0.073 | 66.67 | 1200 | 2.2520 | 0.4131 | 0.301 | 0.3999 | 16.25 | 0.3713 |
78
- | 0.0739 | 69.44 | 1250 | 2.2602 | 0.3582 | 0.2145 | 0.3426 | 17.0 | 0.3051 |
79
- | 0.0799 | 72.22 | 1300 | 2.3278 | 0.369 | 0.2242 | 0.3534 | 16.75 | 0.3156 |
80
- | 0.0546 | 75.0 | 1350 | 2.4021 | 0.369 | 0.2242 | 0.3559 | 16.5 | 0.3164 |
81
- | 0.0674 | 77.78 | 1400 | 2.3493 | 0.4149 | 0.2924 | 0.4017 | 17.25 | 0.3697 |
82
- | 0.0459 | 80.56 | 1450 | 2.3503 | 0.426 | 0.3153 | 0.4104 | 16.125 | 0.3839 |
83
- | 0.0501 | 83.33 | 1500 | 2.3719 | 0.4172 | 0.301 | 0.4016 | 15.375 | 0.3732 |
84
- | 0.0509 | 86.11 | 1550 | 2.4419 | 0.4361 | 0.3188 | 0.4229 | 16.375 | 0.3926 |
85
- | 0.0449 | 88.89 | 1600 | 2.3172 | 0.4514 | 0.3188 | 0.4375 | 16.375 | 0.4026 |
86
- | 0.0408 | 91.67 | 1650 | 2.4438 | 0.4349 | 0.3153 | 0.4217 | 16.25 | 0.3906 |
87
- | 0.0357 | 94.44 | 1700 | 2.5406 | 0.4236 | 0.3153 | 0.4104 | 16.25 | 0.3831 |
88
- | 0.0403 | 97.22 | 1750 | 2.4441 | 0.4111 | 0.3153 | 0.398 | 16.375 | 0.3748 |
89
- | 0.0489 | 100.0 | 1800 | 2.4599 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 |
90
- | 0.032 | 102.78 | 1850 | 2.6235 | 0.4515 | 0.3335 | 0.4359 | 15.0 | 0.407 |
91
- | 0.0379 | 105.56 | 1900 | 2.6058 | 0.4515 | 0.3335 | 0.4359 | 15.125 | 0.407 |
92
- | 0.0466 | 108.33 | 1950 | 2.5748 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 |
93
- | 0.0317 | 111.11 | 2000 | 2.6638 | 0.4169 | 0.3153 | 0.4013 | 16.125 | 0.3778 |
94
- | 0.0234 | 113.89 | 2050 | 2.7407 | 0.4334 | 0.3153 | 0.4178 | 15.5 | 0.3888 |
95
- | 0.0308 | 116.67 | 2100 | 2.7086 | 0.4201 | 0.3153 | 0.4044 | 16.125 | 0.3799 |
96
- | 0.0305 | 119.44 | 2150 | 2.7068 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
97
- | 0.0289 | 122.22 | 2200 | 2.8503 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
98
- | 0.0555 | 125.0 | 2250 | 2.8522 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
99
- | 0.022 | 127.78 | 2300 | 2.9057 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
100
- | 0.0369 | 130.56 | 2350 | 2.8736 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
101
- | 0.0195 | 133.33 | 2400 | 2.7637 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
102
- | 0.0387 | 136.11 | 2450 | 2.7437 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
103
- | 0.0298 | 138.89 | 2500 | 2.8818 | 0.391 | 0.2665 | 0.3754 | 16.25 | 0.3443 |
104
- | 0.0265 | 141.67 | 2550 | 2.8340 | 0.3776 | 0.2665 | 0.362 | 16.5 | 0.3353 |
105
- | 0.0182 | 144.44 | 2600 | 2.8739 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
106
 
107
 
108
  ### Framework versions
 
14
 
15
  This model is a fine-tuned version of [cointegrated/rut5-base-absum](https://huggingface.co/cointegrated/rut5-base-absum) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
+ - Loss: 1.4464
18
+ - Rouge-1: 0.5076
19
+ - Rouge-2: 0.3897
20
+ - Rouge-l: 0.4945
21
+ - Gen Len: 15.75
22
+ - Avg Rouge F: 0.4639
23
 
24
  ## Model description
25
 
 
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: linear
47
  - lr_scheduler_warmup_steps: 200
48
+ - num_epochs: 100
49
 
50
  ### Training results
51
 
52
+ | Training Loss | Epoch | Step | Validation Loss | Rouge-1 | Rouge-2 | Rouge-l | Gen Len | Avg Rouge F |
53
+ |:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:-------:|:-----------:|
54
+ | 2.6017 | 2.78 | 50 | 2.0030 | 0.0 | 0.0 | 0.0 | 8.125 | 0.0 |
55
+ | 2.1413 | 5.56 | 100 | 1.5154 | 0.1125 | 0.0317 | 0.0958 | 11.5 | 0.08 |
56
+ | 1.6874 | 8.33 | 150 | 1.2364 | 0.3417 | 0.2312 | 0.325 | 13.25 | 0.2993 |
57
+ | 1.2272 | 11.11 | 200 | 1.1259 | 0.3605 | 0.2437 | 0.3291 | 14.25 | 0.3111 |
58
+ | 0.9384 | 13.89 | 250 | 1.0853 | 0.4505 | 0.3 | 0.4211 | 13.5 | 0.3905 |
59
+ | 0.7071 | 16.67 | 300 | 1.0607 | 0.3559 | 0.1368 | 0.3133 | 14.875 | 0.2687 |
60
+ | 0.5871 | 19.44 | 350 | 1.0346 | 0.5377 | 0.4194 | 0.5126 | 16.0 | 0.4899 |
61
+ | 0.4194 | 22.22 | 400 | 1.0672 | 0.5079 | 0.3819 | 0.4829 | 15.5 | 0.4576 |
62
+ | 0.3685 | 25.0 | 450 | 1.1284 | 0.5029 | 0.3835 | 0.4897 | 14.75 | 0.4587 |
63
+ | 0.2884 | 27.78 | 500 | 1.1729 | 0.5427 | 0.421 | 0.5164 | 15.875 | 0.4933 |
64
+ | 0.2368 | 30.56 | 550 | 1.1640 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 |
65
+ | 0.195 | 33.33 | 600 | 1.2053 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 |
66
+ | 0.1667 | 36.11 | 650 | 1.2525 | 0.4245 | 0.2717 | 0.4114 | 16.125 | 0.3692 |
67
+ | 0.1491 | 38.89 | 700 | 1.3346 | 0.5032 | 0.3897 | 0.4901 | 16.0 | 0.461 |
68
+ | 0.1122 | 41.67 | 750 | 1.3354 | 0.5094 | 0.4062 | 0.5094 | 15.375 | 0.475 |
69
+ | 0.1166 | 44.44 | 800 | 1.3685 | 0.5076 | 0.3897 | 0.4945 | 15.625 | 0.4639 |
70
+ | 0.0973 | 47.22 | 850 | 1.4157 | 0.5076 | 0.3897 | 0.4945 | 15.375 | 0.4639 |
71
+ | 0.0944 | 50.0 | 900 | 1.4523 | 0.5095 | 0.3897 | 0.4963 | 15.125 | 0.4652 |
72
+ | 0.0744 | 52.78 | 950 | 1.4221 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 |
73
+ | 0.0745 | 55.56 | 1000 | 1.4464 | 0.5076 | 0.3897 | 0.4945 | 15.75 | 0.4639 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
74
 
75
 
76
  ### Framework versions