dCOS/lilt_out_4

output_LiLT_test_04

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1349
• Precision: 0.7800
• Recall: 0.7950
• F1: 0.7874
• Accuracy: 0.9620

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 300
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.0808	100	0.5003	0.1450	0.0360	0.0577	0.8851
No log	0.1617	200	0.3188	0.4026	0.4142	0.4084	0.9031
No log	0.2425	300	0.2046	0.6217	0.5907	0.6058	0.9355
No log	0.3234	400	0.2054	0.6696	0.6647	0.6671	0.9404
0.3884	0.4042	500	0.1591	0.6765	0.7486	0.7107	0.9505
0.3884	0.4850	600	0.1795	0.6579	0.7391	0.6962	0.9394
0.3884	0.5659	700	0.1602	0.6660	0.8193	0.7347	0.9461
0.3884	0.6467	800	0.1879	0.6634	0.8129	0.7306	0.9439
0.3884	0.7276	900	0.1450	0.7756	0.8040	0.7895	0.9606
0.0964	0.8084	1000	0.1399	0.7652	0.7901	0.7775	0.9591
0.0964	0.8892	1100	0.1483	0.7767	0.7743	0.7755	0.9574
0.0964	0.9701	1200	0.1513	0.8295	0.7642	0.7955	0.9618
0.0964	1.0509	1300	0.1395	0.8197	0.7861	0.8025	0.9644
0.0964	1.1318	1400	0.1397	0.7436	0.8149	0.7776	0.9599
0.0762	1.2126	1500	0.1244	0.7928	0.8040	0.7983	0.9627
0.0762	1.2935	1600	0.1388	0.7541	0.8397	0.7946	0.9610
0.0762	1.3743	1700	0.1242	0.8207	0.8037	0.8121	0.9660
0.0762	1.4551	1800	0.1429	0.7529	0.8334	0.7911	0.9589
0.0762	1.5360	1900	0.1349	0.7800	0.7950	0.7874	0.9620

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1