FNST_trad_l

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.0024
• Accuracy: 0.5983
• F1: 0.5955

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 42

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.127	1.0	2000	1.1060	0.5133	0.4590
0.9786	2.0	4000	1.0075	0.5642	0.5524
0.9072	3.0	6000	0.9958	0.5733	0.5697
0.869	4.0	8000	0.9776	0.5917	0.5957
0.8243	5.0	10000	0.9760	0.5817	0.5860
0.7659	6.0	12000	0.9826	0.595	0.5993
0.7414	7.0	14000	1.0055	0.5933	0.6001
0.7023	8.0	16000	1.0113	0.5908	0.5959
0.6745	9.0	18000	1.0527	0.5933	0.5949
0.6161	10.0	20000	1.1227	0.5883	0.5920
0.5863	11.0	22000	1.1571	0.5883	0.5896
0.5406	12.0	24000	1.1883	0.5908	0.5954
0.5185	13.0	26000	1.2686	0.5917	0.5957
0.4796	14.0	28000	1.3313	0.5992	0.6067
0.4379	15.0	30000	1.4234	0.595	0.5970
0.3883	16.0	32000	1.5582	0.5958	0.5994
0.3934	17.0	34000	1.6591	0.595	0.6012
0.359	18.0	36000	1.8129	0.595	0.6011
0.3249	19.0	38000	1.9811	0.5917	0.5966
0.2954	20.0	40000	2.1860	0.5858	0.5901
0.3064	21.0	42000	2.2548	0.5858	0.5904
0.2844	22.0	44000	2.3557	0.6	0.6045
0.2471	23.0	46000	2.5137	0.6017	0.6033
0.2432	24.0	48000	2.6458	0.5992	0.6035
0.2247	25.0	50000	2.8667	0.5983	0.6026
0.213	26.0	52000	2.8895	0.6042	0.6088
0.1792	27.0	54000	3.0338	0.6008	0.6062
0.1723	28.0	56000	3.1234	0.5975	0.6008
0.1562	29.0	58000	3.2822	0.5942	0.5992
0.1437	30.0	60000	3.3156	0.6067	0.6096
0.151	31.0	62000	3.3923	0.6075	0.6098
0.1446	32.0	64000	3.4562	0.6058	0.6084
0.109	33.0	66000	3.6100	0.6017	0.6038
0.1138	34.0	68000	3.6468	0.6083	0.6078
0.1068	35.0	70000	3.6758	0.5967	0.6004
0.1133	36.0	72000	3.7340	0.6033	0.6025
0.0865	37.0	74000	3.7666	0.6083	0.6079
0.0936	38.0	76000	3.8650	0.6058	0.6073
0.0957	39.0	78000	3.9209	0.605	0.6025
0.1027	40.0	80000	3.9211	0.6058	0.6054
0.0828	41.0	82000	4.0124	0.6058	0.6009
0.0742	42.0	84000	4.0024	0.5983	0.5955

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1