bert-base-cased-samsum

This model is a fine-tuned version of on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 2.7369
• Rouge1: 34.9636
• Rouge2: 10.6358
• Rougel: 27.6003
• Rougelsum: 30.9654
• Gen Len: 17.6020

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 36
• eval_batch_size: 36
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	410	2.9218	29.2936	7.4008	23.9609	26.3194	17.2332
2.6834	2.0	820	2.7635	31.9826	8.9758	26.1311	28.7458	16.9866
2.3529	3.0	1230	2.7369	34.9636	10.6358	27.6003	30.9654	17.6020
1.9608	4.0	1640	2.7711	35.8322	11.3676	29.0276	32.2881	16.9133
1.6459	5.0	2050	2.7832	36.8688	11.8883	29.3721	32.8683	17.0879
1.6459	6.0	2460	2.8334	36.489	11.5372	29.2263	32.5406	17.8901
1.3791	7.0	2870	2.8767	37.0743	11.8554	29.4063	32.7543	17.6093
1.1687	8.0	3280	2.9232	37.2	11.8723	29.5194	32.9481	17.6581
1.0249	9.0	3690	2.9456	37.1872	12.0958	29.621	33.0073	17.8840
0.9259	10.0	4100	2.9719	37.1213	12.1068	29.5138	33.0372	17.8278

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1