metadata
license: mit
tags:
- generated_from_trainer
model-index:
- name: rut5-base-absum-tech-support-calls
results: []
rut5-base-absum-tech-support-calls
This model is a fine-tuned version of cointegrated/rut5-base-absum on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.8739
- Rouge-1: 0.4059
- Rouge-2: 0.2831
- Rouge-l: 0.3902
- Gen Len: 15.5
- Avg Rouge F: 0.3598
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 3
- eval_batch_size: 1
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 250
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge-1 | Rouge-2 | Rouge-l | Gen Len | Avg Rouge F |
|---|---|---|---|---|---|---|---|---|
| 2.7022 | 2.78 | 50 | 2.2970 | 0.0 | 0.0 | 0.0 | 6.875 | 0.0 |
| 2.2932 | 5.56 | 100 | 1.8183 | 0.0 | 0.0 | 0.0 | 10.375 | 0.0 |
| 1.8234 | 8.33 | 150 | 1.4890 | 0.3588 | 0.2205 | 0.3262 | 14.0 | 0.3018 |
| 1.3727 | 11.11 | 200 | 1.3740 | 0.3493 | 0.1653 | 0.3167 | 12.375 | 0.2771 |
| 1.0367 | 13.89 | 250 | 1.3833 | 0.2607 | 0.0984 | 0.2331 | 15.375 | 0.1974 |
| 0.841 | 16.67 | 300 | 1.3516 | 0.3713 | 0.1857 | 0.3594 | 16.0 | 0.3055 |
| 0.7182 | 19.44 | 350 | 1.3607 | 0.3352 | 0.143 | 0.3233 | 16.125 | 0.2672 |
| 0.5102 | 22.22 | 400 | 1.3673 | 0.36 | 0.1597 | 0.3349 | 16.625 | 0.2849 |
| 0.4595 | 25.0 | 450 | 1.3715 | 0.3892 | 0.2153 | 0.3641 | 17.125 | 0.3228 |
| 0.3886 | 27.78 | 500 | 1.4634 | 0.3801 | 0.2274 | 0.3682 | 16.375 | 0.3252 |
| 0.3158 | 30.56 | 550 | 1.5124 | 0.3938 | 0.2319 | 0.3672 | 16.75 | 0.331 |
| 0.2687 | 33.33 | 600 | 1.5868 | 0.3987 | 0.2568 | 0.3848 | 16.5 | 0.3468 |
| 0.2361 | 36.11 | 650 | 1.6460 | 0.375 | 0.2107 | 0.3631 | 17.75 | 0.3163 |
| 0.1991 | 38.89 | 700 | 1.6947 | 0.3605 | 0.2177 | 0.3474 | 16.25 | 0.3085 |
| 0.151 | 41.67 | 750 | 1.8248 | 0.3832 | 0.2274 | 0.3559 | 16.5 | 0.3222 |
| 0.1517 | 44.44 | 800 | 1.7884 | 0.4309 | 0.294 | 0.4184 | 16.875 | 0.3811 |
| 0.1444 | 47.22 | 850 | 1.8519 | 0.3843 | 0.2107 | 0.3711 | 17.125 | 0.322 |
| 0.1106 | 50.0 | 900 | 1.9637 | 0.383 | 0.2107 | 0.3691 | 17.5 | 0.3209 |
| 0.0961 | 52.78 | 950 | 2.0718 | 0.3645 | 0.2177 | 0.3488 | 16.75 | 0.3103 |
| 0.1131 | 55.56 | 1000 | 1.9935 | 0.3602 | 0.2153 | 0.3446 | 16.75 | 0.3067 |
| 0.0996 | 58.33 | 1050 | 2.0616 | 0.4153 | 0.2986 | 0.3996 | 16.0 | 0.3712 |
| 0.0663 | 61.11 | 1100 | 2.1466 | 0.4257 | 0.301 | 0.409 | 14.625 | 0.3786 |
| 0.0789 | 63.89 | 1150 | 2.1657 | 0.4166 | 0.301 | 0.4009 | 16.0 | 0.3728 |
| 0.073 | 66.67 | 1200 | 2.2520 | 0.4131 | 0.301 | 0.3999 | 16.25 | 0.3713 |
| 0.0739 | 69.44 | 1250 | 2.2602 | 0.3582 | 0.2145 | 0.3426 | 17.0 | 0.3051 |
| 0.0799 | 72.22 | 1300 | 2.3278 | 0.369 | 0.2242 | 0.3534 | 16.75 | 0.3156 |
| 0.0546 | 75.0 | 1350 | 2.4021 | 0.369 | 0.2242 | 0.3559 | 16.5 | 0.3164 |
| 0.0674 | 77.78 | 1400 | 2.3493 | 0.4149 | 0.2924 | 0.4017 | 17.25 | 0.3697 |
| 0.0459 | 80.56 | 1450 | 2.3503 | 0.426 | 0.3153 | 0.4104 | 16.125 | 0.3839 |
| 0.0501 | 83.33 | 1500 | 2.3719 | 0.4172 | 0.301 | 0.4016 | 15.375 | 0.3732 |
| 0.0509 | 86.11 | 1550 | 2.4419 | 0.4361 | 0.3188 | 0.4229 | 16.375 | 0.3926 |
| 0.0449 | 88.89 | 1600 | 2.3172 | 0.4514 | 0.3188 | 0.4375 | 16.375 | 0.4026 |
| 0.0408 | 91.67 | 1650 | 2.4438 | 0.4349 | 0.3153 | 0.4217 | 16.25 | 0.3906 |
| 0.0357 | 94.44 | 1700 | 2.5406 | 0.4236 | 0.3153 | 0.4104 | 16.25 | 0.3831 |
| 0.0403 | 97.22 | 1750 | 2.4441 | 0.4111 | 0.3153 | 0.398 | 16.375 | 0.3748 |
| 0.0489 | 100.0 | 1800 | 2.4599 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 |
| 0.032 | 102.78 | 1850 | 2.6235 | 0.4515 | 0.3335 | 0.4359 | 15.0 | 0.407 |
| 0.0379 | 105.56 | 1900 | 2.6058 | 0.4515 | 0.3335 | 0.4359 | 15.125 | 0.407 |
| 0.0466 | 108.33 | 1950 | 2.5748 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 |
| 0.0317 | 111.11 | 2000 | 2.6638 | 0.4169 | 0.3153 | 0.4013 | 16.125 | 0.3778 |
| 0.0234 | 113.89 | 2050 | 2.7407 | 0.4334 | 0.3153 | 0.4178 | 15.5 | 0.3888 |
| 0.0308 | 116.67 | 2100 | 2.7086 | 0.4201 | 0.3153 | 0.4044 | 16.125 | 0.3799 |
| 0.0305 | 119.44 | 2150 | 2.7068 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0289 | 122.22 | 2200 | 2.8503 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0555 | 125.0 | 2250 | 2.8522 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.022 | 127.78 | 2300 | 2.9057 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0369 | 130.56 | 2350 | 2.8736 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0195 | 133.33 | 2400 | 2.7637 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0387 | 136.11 | 2450 | 2.7437 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
| 0.0298 | 138.89 | 2500 | 2.8818 | 0.391 | 0.2665 | 0.3754 | 16.25 | 0.3443 |
| 0.0265 | 141.67 | 2550 | 2.8340 | 0.3776 | 0.2665 | 0.362 | 16.5 | 0.3353 |
| 0.0182 | 144.44 | 2600 | 2.8739 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 |
Framework versions
- Transformers 4.29.2
- Pytorch 2.0.1+cu118
- Tokenizers 0.13.3