--- license: mit tags: - generated_from_trainer model-index: - name: rut5-base-absum-tech-support-calls results: [] --- # rut5-base-absum-tech-support-calls This model is a fine-tuned version of [cointegrated/rut5-base-absum](https://huggingface.co/cointegrated/rut5-base-absum) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 2.8739 - Rouge-1: 0.4059 - Rouge-2: 0.2831 - Rouge-l: 0.3902 - Gen Len: 15.5 - Avg Rouge F: 0.3598 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 3 - eval_batch_size: 1 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 200 - num_epochs: 250 ### Training results | Training Loss | Epoch | Step | Validation Loss | Rouge-1 | Rouge-2 | Rouge-l | Gen Len | Avg Rouge F | |:-------------:|:------:|:----:|:---------------:|:-------:|:-------:|:-------:|:-------:|:-----------:| | 2.7022 | 2.78 | 50 | 2.2970 | 0.0 | 0.0 | 0.0 | 6.875 | 0.0 | | 2.2932 | 5.56 | 100 | 1.8183 | 0.0 | 0.0 | 0.0 | 10.375 | 0.0 | | 1.8234 | 8.33 | 150 | 1.4890 | 0.3588 | 0.2205 | 0.3262 | 14.0 | 0.3018 | | 1.3727 | 11.11 | 200 | 1.3740 | 0.3493 | 0.1653 | 0.3167 | 12.375 | 0.2771 | | 1.0367 | 13.89 | 250 | 1.3833 | 0.2607 | 0.0984 | 0.2331 | 15.375 | 0.1974 | | 0.841 | 16.67 | 300 | 1.3516 | 0.3713 | 0.1857 | 0.3594 | 16.0 | 0.3055 | | 0.7182 | 19.44 | 350 | 1.3607 | 0.3352 | 0.143 | 0.3233 | 16.125 | 0.2672 | | 0.5102 | 22.22 | 400 | 1.3673 | 0.36 | 0.1597 | 0.3349 | 16.625 | 0.2849 | | 0.4595 | 25.0 | 450 | 1.3715 | 0.3892 | 0.2153 | 0.3641 | 17.125 | 0.3228 | | 0.3886 | 27.78 | 500 | 1.4634 | 0.3801 | 0.2274 | 0.3682 | 16.375 | 0.3252 | | 0.3158 | 30.56 | 550 | 1.5124 | 0.3938 | 0.2319 | 0.3672 | 16.75 | 0.331 | | 0.2687 | 33.33 | 600 | 1.5868 | 0.3987 | 0.2568 | 0.3848 | 16.5 | 0.3468 | | 0.2361 | 36.11 | 650 | 1.6460 | 0.375 | 0.2107 | 0.3631 | 17.75 | 0.3163 | | 0.1991 | 38.89 | 700 | 1.6947 | 0.3605 | 0.2177 | 0.3474 | 16.25 | 0.3085 | | 0.151 | 41.67 | 750 | 1.8248 | 0.3832 | 0.2274 | 0.3559 | 16.5 | 0.3222 | | 0.1517 | 44.44 | 800 | 1.7884 | 0.4309 | 0.294 | 0.4184 | 16.875 | 0.3811 | | 0.1444 | 47.22 | 850 | 1.8519 | 0.3843 | 0.2107 | 0.3711 | 17.125 | 0.322 | | 0.1106 | 50.0 | 900 | 1.9637 | 0.383 | 0.2107 | 0.3691 | 17.5 | 0.3209 | | 0.0961 | 52.78 | 950 | 2.0718 | 0.3645 | 0.2177 | 0.3488 | 16.75 | 0.3103 | | 0.1131 | 55.56 | 1000 | 1.9935 | 0.3602 | 0.2153 | 0.3446 | 16.75 | 0.3067 | | 0.0996 | 58.33 | 1050 | 2.0616 | 0.4153 | 0.2986 | 0.3996 | 16.0 | 0.3712 | | 0.0663 | 61.11 | 1100 | 2.1466 | 0.4257 | 0.301 | 0.409 | 14.625 | 0.3786 | | 0.0789 | 63.89 | 1150 | 2.1657 | 0.4166 | 0.301 | 0.4009 | 16.0 | 0.3728 | | 0.073 | 66.67 | 1200 | 2.2520 | 0.4131 | 0.301 | 0.3999 | 16.25 | 0.3713 | | 0.0739 | 69.44 | 1250 | 2.2602 | 0.3582 | 0.2145 | 0.3426 | 17.0 | 0.3051 | | 0.0799 | 72.22 | 1300 | 2.3278 | 0.369 | 0.2242 | 0.3534 | 16.75 | 0.3156 | | 0.0546 | 75.0 | 1350 | 2.4021 | 0.369 | 0.2242 | 0.3559 | 16.5 | 0.3164 | | 0.0674 | 77.78 | 1400 | 2.3493 | 0.4149 | 0.2924 | 0.4017 | 17.25 | 0.3697 | | 0.0459 | 80.56 | 1450 | 2.3503 | 0.426 | 0.3153 | 0.4104 | 16.125 | 0.3839 | | 0.0501 | 83.33 | 1500 | 2.3719 | 0.4172 | 0.301 | 0.4016 | 15.375 | 0.3732 | | 0.0509 | 86.11 | 1550 | 2.4419 | 0.4361 | 0.3188 | 0.4229 | 16.375 | 0.3926 | | 0.0449 | 88.89 | 1600 | 2.3172 | 0.4514 | 0.3188 | 0.4375 | 16.375 | 0.4026 | | 0.0408 | 91.67 | 1650 | 2.4438 | 0.4349 | 0.3153 | 0.4217 | 16.25 | 0.3906 | | 0.0357 | 94.44 | 1700 | 2.5406 | 0.4236 | 0.3153 | 0.4104 | 16.25 | 0.3831 | | 0.0403 | 97.22 | 1750 | 2.4441 | 0.4111 | 0.3153 | 0.398 | 16.375 | 0.3748 | | 0.0489 | 100.0 | 1800 | 2.4599 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 | | 0.032 | 102.78 | 1850 | 2.6235 | 0.4515 | 0.3335 | 0.4359 | 15.0 | 0.407 | | 0.0379 | 105.56 | 1900 | 2.6058 | 0.4515 | 0.3335 | 0.4359 | 15.125 | 0.407 | | 0.0466 | 108.33 | 1950 | 2.5748 | 0.4154 | 0.3153 | 0.3997 | 16.125 | 0.3768 | | 0.0317 | 111.11 | 2000 | 2.6638 | 0.4169 | 0.3153 | 0.4013 | 16.125 | 0.3778 | | 0.0234 | 113.89 | 2050 | 2.7407 | 0.4334 | 0.3153 | 0.4178 | 15.5 | 0.3888 | | 0.0308 | 116.67 | 2100 | 2.7086 | 0.4201 | 0.3153 | 0.4044 | 16.125 | 0.3799 | | 0.0305 | 119.44 | 2150 | 2.7068 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0289 | 122.22 | 2200 | 2.8503 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0555 | 125.0 | 2250 | 2.8522 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.022 | 127.78 | 2300 | 2.9057 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0369 | 130.56 | 2350 | 2.8736 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0195 | 133.33 | 2400 | 2.7637 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0387 | 136.11 | 2450 | 2.7437 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | | 0.0298 | 138.89 | 2500 | 2.8818 | 0.391 | 0.2665 | 0.3754 | 16.25 | 0.3443 | | 0.0265 | 141.67 | 2550 | 2.8340 | 0.3776 | 0.2665 | 0.362 | 16.5 | 0.3353 | | 0.0182 | 144.44 | 2600 | 2.8739 | 0.4059 | 0.2831 | 0.3902 | 15.5 | 0.3598 | ### Framework versions - Transformers 4.29.2 - Pytorch 2.0.1+cu118 - Tokenizers 0.13.3