flan-t5-base-model3-token1

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1761
• Rouge1: 74.8442
• Rouge2: 67.1345
• Rougel: 73.9377
• Rougelsum: 74.7805
• Gen Len: 16.0557

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
11.3452	0.7	200	0.4023	54.9094	45.9641	54.4888	54.7708	8.6307
0.3215	1.39	400	0.2252	72.8643	63.763	71.6779	72.5956	16.2108
0.2502	2.09	600	0.2052	73.7732	65.343	72.7147	73.7003	16.0035
0.2072	2.79	800	0.1977	73.7375	65.4023	72.5708	73.5249	16.0436
0.191	3.48	1000	0.1923	74.1851	65.5192	73.2189	74.0127	15.8502
0.1913	4.18	1200	0.1870	74.3015	66.4095	73.113	74.1052	16.0366
0.1672	4.88	1400	0.1814	74.5867	66.657	73.6986	74.4877	15.9652
0.1666	5.57	1600	0.1818	74.4879	66.4071	73.3689	74.3287	15.9739
0.1565	6.27	1800	0.1774	74.4269	66.8322	73.4838	74.3445	16.0906
0.1533	6.97	2000	0.1755	74.7965	66.8688	73.522	74.6461	15.9373
0.1447	7.67	2200	0.1764	74.4052	66.6402	73.3189	74.2453	15.9774
0.141	8.36	2400	0.1768	74.6052	66.5586	73.4595	74.3578	15.9338
0.1395	9.06	2600	0.1762	74.739	66.8901	73.6772	74.5243	16.0697
0.136	9.76	2800	0.1761	74.8442	67.1345	73.9377	74.7805	16.0557

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.14.0