abletobetable's picture
update model card README.md
b25a0d0
metadata
license: mit
tags:
  - generated_from_trainer
model-index:
  - name: rut5-base-absum-tech-support-calls
    results: []

rut5-base-absum-tech-support-calls

This model is a fine-tuned version of cointegrated/rut5-base-absum on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4464
  • Rouge-1: 0.5076
  • Rouge-2: 0.3897
  • Rouge-l: 0.4945
  • Gen Len: 15.75
  • Avg Rouge F: 0.4639

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 3
  • eval_batch_size: 1
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 200
  • num_epochs: 100

Training results

Training Loss Epoch Step Validation Loss Rouge-1 Rouge-2 Rouge-l Gen Len Avg Rouge F
2.6017 2.78 50 2.0030 0.0 0.0 0.0 8.125 0.0
2.1413 5.56 100 1.5154 0.1125 0.0317 0.0958 11.5 0.08
1.6874 8.33 150 1.2364 0.3417 0.2312 0.325 13.25 0.2993
1.2272 11.11 200 1.1259 0.3605 0.2437 0.3291 14.25 0.3111
0.9384 13.89 250 1.0853 0.4505 0.3 0.4211 13.5 0.3905
0.7071 16.67 300 1.0607 0.3559 0.1368 0.3133 14.875 0.2687
0.5871 19.44 350 1.0346 0.5377 0.4194 0.5126 16.0 0.4899
0.4194 22.22 400 1.0672 0.5079 0.3819 0.4829 15.5 0.4576
0.3685 25.0 450 1.1284 0.5029 0.3835 0.4897 14.75 0.4587
0.2884 27.78 500 1.1729 0.5427 0.421 0.5164 15.875 0.4933
0.2368 30.56 550 1.1640 0.5326 0.421 0.5195 15.25 0.491
0.195 33.33 600 1.2053 0.5326 0.421 0.5195 15.25 0.491
0.1667 36.11 650 1.2525 0.4245 0.2717 0.4114 16.125 0.3692
0.1491 38.89 700 1.3346 0.5032 0.3897 0.4901 16.0 0.461
0.1122 41.67 750 1.3354 0.5094 0.4062 0.5094 15.375 0.475
0.1166 44.44 800 1.3685 0.5076 0.3897 0.4945 15.625 0.4639
0.0973 47.22 850 1.4157 0.5076 0.3897 0.4945 15.375 0.4639
0.0944 50.0 900 1.4523 0.5095 0.3897 0.4963 15.125 0.4652
0.0744 52.78 950 1.4221 0.5326 0.421 0.5195 15.25 0.491
0.0745 55.56 1000 1.4464 0.5076 0.3897 0.4945 15.75 0.4639

Framework versions

  • Transformers 4.29.2
  • Pytorch 2.0.1+cu118
  • Tokenizers 0.13.3