|
--- |
|
license: mit |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: rut5-base-absum-tech-support-calls |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# rut5-base-absum-tech-support-calls |
|
|
|
This model is a fine-tuned version of [cointegrated/rut5-base-absum](https://huggingface.co/cointegrated/rut5-base-absum) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.4464 |
|
- Rouge-1: 0.5076 |
|
- Rouge-2: 0.3897 |
|
- Rouge-l: 0.4945 |
|
- Gen Len: 15.75 |
|
- Avg Rouge F: 0.4639 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 3 |
|
- eval_batch_size: 1 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- lr_scheduler_warmup_steps: 200 |
|
- num_epochs: 100 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Rouge-1 | Rouge-2 | Rouge-l | Gen Len | Avg Rouge F | |
|
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:-------:|:-----------:| |
|
| 2.6017 | 2.78 | 50 | 2.0030 | 0.0 | 0.0 | 0.0 | 8.125 | 0.0 | |
|
| 2.1413 | 5.56 | 100 | 1.5154 | 0.1125 | 0.0317 | 0.0958 | 11.5 | 0.08 | |
|
| 1.6874 | 8.33 | 150 | 1.2364 | 0.3417 | 0.2312 | 0.325 | 13.25 | 0.2993 | |
|
| 1.2272 | 11.11 | 200 | 1.1259 | 0.3605 | 0.2437 | 0.3291 | 14.25 | 0.3111 | |
|
| 0.9384 | 13.89 | 250 | 1.0853 | 0.4505 | 0.3 | 0.4211 | 13.5 | 0.3905 | |
|
| 0.7071 | 16.67 | 300 | 1.0607 | 0.3559 | 0.1368 | 0.3133 | 14.875 | 0.2687 | |
|
| 0.5871 | 19.44 | 350 | 1.0346 | 0.5377 | 0.4194 | 0.5126 | 16.0 | 0.4899 | |
|
| 0.4194 | 22.22 | 400 | 1.0672 | 0.5079 | 0.3819 | 0.4829 | 15.5 | 0.4576 | |
|
| 0.3685 | 25.0 | 450 | 1.1284 | 0.5029 | 0.3835 | 0.4897 | 14.75 | 0.4587 | |
|
| 0.2884 | 27.78 | 500 | 1.1729 | 0.5427 | 0.421 | 0.5164 | 15.875 | 0.4933 | |
|
| 0.2368 | 30.56 | 550 | 1.1640 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 | |
|
| 0.195 | 33.33 | 600 | 1.2053 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 | |
|
| 0.1667 | 36.11 | 650 | 1.2525 | 0.4245 | 0.2717 | 0.4114 | 16.125 | 0.3692 | |
|
| 0.1491 | 38.89 | 700 | 1.3346 | 0.5032 | 0.3897 | 0.4901 | 16.0 | 0.461 | |
|
| 0.1122 | 41.67 | 750 | 1.3354 | 0.5094 | 0.4062 | 0.5094 | 15.375 | 0.475 | |
|
| 0.1166 | 44.44 | 800 | 1.3685 | 0.5076 | 0.3897 | 0.4945 | 15.625 | 0.4639 | |
|
| 0.0973 | 47.22 | 850 | 1.4157 | 0.5076 | 0.3897 | 0.4945 | 15.375 | 0.4639 | |
|
| 0.0944 | 50.0 | 900 | 1.4523 | 0.5095 | 0.3897 | 0.4963 | 15.125 | 0.4652 | |
|
| 0.0744 | 52.78 | 950 | 1.4221 | 0.5326 | 0.421 | 0.5195 | 15.25 | 0.491 | |
|
| 0.0745 | 55.56 | 1000 | 1.4464 | 0.5076 | 0.3897 | 0.4945 | 15.75 | 0.4639 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.29.2 |
|
- Pytorch 2.0.1+cu118 |
|
- Tokenizers 0.13.3 |
|
|
