t5-small-finetune-cnn

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.9579
• Rouge1: 24.7426
• Rouge2: 10.4667
• Rougel: 20.2334
• Rougelsum: 23.0122
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.9721	1.0	2000	1.9608	25.1804	10.8327	20.5778	23.3974	19.0
1.9466	2.0	4000	1.9549	25.0152	10.6784	20.4465	23.2601	19.0
1.8932	3.0	6000	1.9515	25.0464	10.7024	20.3992	23.2249	19.0
1.8564	4.0	8000	1.9489	25.0313	10.642	20.3601	23.2032	19.0
1.862	5.0	10000	1.9510	24.9582	10.614	20.3625	23.1762	19.0
1.8478	6.0	12000	1.9502	25.032	10.7084	20.4506	23.2435	19.0
1.819	7.0	14000	1.9495	24.7874	10.4848	20.2893	23.0832	19.0
1.7869	8.0	16000	1.9470	24.7095	10.4465	20.1705	22.9248	19.0
1.8068	9.0	18000	1.9510	24.705	10.4407	20.1684	22.9817	19.0
1.768	10.0	20000	1.9517	24.6067	10.4281	20.0765	22.9034	19.0
1.7713	11.0	22000	1.9524	24.6871	10.4126	20.1802	22.962	19.0
1.7635	12.0	24000	1.9548	24.5998	10.3969	20.1427	22.9191	19.0
1.7625	13.0	26000	1.9561	24.66	10.4032	20.1732	22.9256	19.0
1.7461	14.0	28000	1.9551	24.7071	10.4209	20.1833	22.9803	19.0
1.7271	15.0	30000	1.9558	24.6682	10.4162	20.198	22.9445	19.0
1.7452	16.0	32000	1.9563	24.8148	10.4558	20.2123	23.0374	19.0
1.7489	17.0	34000	1.9576	24.6459	10.3782	20.1213	22.8918	19.0
1.724	18.0	36000	1.9581	24.7384	10.427	20.2088	22.9971	19.0
1.7236	19.0	38000	1.9581	24.7366	10.4394	20.2028	23.0286	19.0
1.7331	20.0	40000	1.9579	24.7426	10.4667	20.2334	23.0122	19.0

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3