T5_base_title / README.md
Swarnava's picture
End of training
7a7b912 verified
metadata
license: apache-2.0
base_model: t5-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: T5_base_title
    results: []

T5_base_title

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 2.0487
  • Rouge1: 0.3848
  • Rouge2: 0.1901
  • Rougel: 0.3297
  • Rougelsum: 0.3288
  • Gen Len: 16.795

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
No log 1.0 200 2.0966 0.3525 0.165 0.2921 0.2934 17.4
No log 2.0 400 2.0533 0.3706 0.1775 0.3145 0.3147 16.52
2.2032 3.0 600 2.0453 0.3754 0.1867 0.3226 0.3227 16.68
2.2032 4.0 800 2.0383 0.379 0.1887 0.3246 0.3243 16.36
1.8535 5.0 1000 2.0376 0.3849 0.1881 0.3269 0.3267 16.755
1.8535 6.0 1200 2.0416 0.378 0.1792 0.3236 0.3228 16.84
1.8535 7.0 1400 2.0445 0.3805 0.1848 0.3249 0.3248 16.65
1.692 8.0 1600 2.0484 0.3876 0.187 0.3289 0.3285 16.845
1.692 9.0 1800 2.0473 0.3891 0.1912 0.3325 0.332 16.815
1.6276 10.0 2000 2.0487 0.3848 0.1901 0.3297 0.3288 16.795

Framework versions

  • Transformers 4.37.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.17.0
  • Tokenizers 0.15.2