t5-small-finetune-xsum

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.4414
• Rouge1: 29.3266
• Rouge2: 8.4122
• Rougel: 23.086
• Rougelsum: 23.0988
• Gen Len: 18.8112

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.7808	1.0	2000	2.5082	27.7519	7.446	21.7066	21.7226	18.835
2.748	2.0	4000	2.4904	27.9132	7.5749	21.9684	21.9859	18.81
2.6996	3.0	6000	2.4786	28.2767	7.8416	22.1965	22.2152	18.785
2.6992	4.0	8000	2.4694	28.6795	7.9755	22.4116	22.437	18.8275
2.6118	5.0	10000	2.4627	28.6839	7.9493	22.4375	22.4522	18.8075
2.6242	6.0	12000	2.4549	28.8803	8.1118	22.6837	22.6895	18.8169
2.5889	7.0	14000	2.4523	29.0163	8.2553	22.9279	22.9428	18.8281
2.5689	8.0	16000	2.4515	28.9347	8.1521	22.7739	22.7803	18.8169
2.5309	9.0	18000	2.4490	29.1943	8.2996	23.0166	23.005	18.8238
2.5179	10.0	20000	2.4460	29.1816	8.3726	23.0678	23.0622	18.8025
2.5114	11.0	22000	2.4451	29.1586	8.3156	23.0407	23.0485	18.8094
2.4775	12.0	24000	2.4440	29.2132	8.452	23.0056	23.0021	18.8069
2.5082	13.0	26000	2.4440	29.1495	8.3541	22.9148	22.9349	18.8025
2.4888	14.0	28000	2.4431	29.2776	8.3071	23.0654	23.0685	18.8138
2.479	15.0	30000	2.4431	29.378	8.4205	23.1346	23.1347	18.8044
2.4464	16.0	32000	2.4427	29.3569	8.4209	23.0688	23.0814	18.8038
2.4431	17.0	34000	2.4423	29.2736	8.3856	23.0737	23.0696	18.8188
2.447	18.0	36000	2.4419	29.2725	8.41	23.0817	23.1089	18.8125
2.4626	19.0	38000	2.4416	29.3144	8.3858	23.0861	23.0993	18.8075
2.4362	20.0	40000	2.4414	29.3266	8.4122	23.086	23.0988	18.8112

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3