text-to-text

This model is a fine-tuned version of google/flan-t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 5.1157
• Rouge1: 39.2425
• Rouge2: 15.3791
• Rougel: 32.2803
• Rougelsum: 35.7848
• Gen Len: 17.5447

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.4195	1.0	287	2.0451	38.2732	15.4594	32.1587	34.1054	14.9350
1.5202	2.0	574	2.1663	37.2747	14.3187	30.7608	33.1298	15.6179
0.8321	3.0	861	2.3894	39.7659	15.9607	32.553	35.5703	15.9837
0.8915	4.0	1148	2.7800	37.5833	14.0917	29.9648	33.7803	17.4472
0.4739	5.0	1435	2.9443	38.6304	14.5346	31.2649	34.7004	16.6260
0.4026	6.0	1722	3.4132	38.5667	13.5102	31.2463	34.3462	17.6016
0.0301	7.0	2009	3.9872	38.4039	14.3773	30.8458	34.5474	17.4959
0.0504	8.0	2296	4.3657	38.2491	13.9452	30.3137	33.6681	17.1870
0.0017	9.0	2583	5.0423	39.4347	14.464	31.561	35.0523	17.1382
0.0038	10.0	2870	5.1157	39.2425	15.3791	32.2803	35.7848	17.5447

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0a0+32f93b1
• Datasets 2.15.0
• Tokenizers 0.15.0