FNST_trad_2a

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.0162
• Accuracy: 0.6525
• F1: 0.6433

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 6
• eval_batch_size: 6
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 78

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.1219	1.0	2000	1.0718	0.5383	0.4455
0.9768	2.0	4000	0.9417	0.595	0.5797
0.8945	3.0	6000	0.9033	0.6092	0.6039
0.8746	4.0	8000	0.8784	0.6275	0.6224
0.802	5.0	10000	0.8671	0.6367	0.6302
0.7519	6.0	12000	0.8689	0.6375	0.6322
0.7418	7.0	14000	0.8705	0.6383	0.6324
0.7059	8.0	16000	0.8827	0.6408	0.6379
0.669	9.0	18000	0.9116	0.635	0.6312
0.6187	10.0	20000	0.9554	0.6483	0.6437
0.5805	11.0	22000	0.9859	0.6383	0.6344
0.5586	12.0	24000	0.9893	0.6425	0.6329
0.4964	13.0	26000	1.0624	0.645	0.6348
0.4637	14.0	28000	1.1429	0.6375	0.6298
0.4146	15.0	30000	1.2550	0.635	0.6291
0.3859	16.0	32000	1.2847	0.6425	0.6350
0.3613	17.0	34000	1.4032	0.6442	0.6375
0.3566	18.0	36000	1.5330	0.6442	0.6363
0.3014	19.0	38000	1.6704	0.6458	0.6384
0.3236	20.0	40000	1.8629	0.6442	0.6406
0.3181	21.0	42000	1.9695	0.6375	0.6361
0.258	22.0	44000	2.0728	0.6467	0.6365
0.2174	23.0	46000	2.1817	0.6442	0.6336
0.2373	24.0	48000	2.3151	0.6542	0.6461
0.2237	25.0	50000	2.3966	0.6392	0.6311
0.1968	26.0	52000	2.5837	0.6375	0.6304
0.1952	27.0	54000	2.6898	0.635	0.6283
0.1734	28.0	56000	2.7676	0.6525	0.6464
0.1791	29.0	58000	2.8467	0.6408	0.6341
0.1723	30.0	60000	2.9415	0.6483	0.6382
0.1437	31.0	62000	2.9579	0.6467	0.6390
0.1377	32.0	64000	3.0478	0.6492	0.6429
0.125	33.0	66000	3.1053	0.6433	0.6333
0.1142	34.0	68000	3.1841	0.6442	0.6371
0.1064	35.0	70000	3.2318	0.6483	0.6414
0.1083	36.0	72000	3.3547	0.6367	0.6271
0.0729	37.0	74000	3.4056	0.6483	0.6393
0.0805	38.0	76000	3.3959	0.6467	0.6396
0.0809	39.0	78000	3.4675	0.6458	0.6390
0.0792	40.0	80000	3.5613	0.6408	0.6370
0.0735	41.0	82000	3.5786	0.6442	0.6367
0.0753	42.0	84000	3.6967	0.6408	0.6320
0.0661	43.0	86000	3.6580	0.6425	0.6380
0.0566	44.0	88000	3.7266	0.6392	0.6320
0.0617	45.0	90000	3.5621	0.6608	0.6543
0.0535	46.0	92000	3.6820	0.6458	0.6350
0.0593	47.0	94000	3.5833	0.6517	0.6438
0.0666	48.0	96000	3.5367	0.6542	0.6465
0.0589	49.0	98000	3.7562	0.6492	0.6438
0.0504	50.0	100000	3.6989	0.6483	0.6372
0.0414	51.0	102000	3.6851	0.6542	0.6472
0.0454	52.0	104000	3.8027	0.6483	0.6436
0.0421	53.0	106000	3.9190	0.6475	0.6415
0.0422	54.0	108000	3.7929	0.6567	0.6478
0.0476	55.0	110000	3.9425	0.6458	0.6387
0.0539	56.0	112000	3.8677	0.6542	0.6477
0.0471	57.0	114000	3.8409	0.6467	0.6390
0.0466	58.0	116000	3.8810	0.6442	0.6394
0.0241	59.0	118000	3.9288	0.645	0.6355
0.0517	60.0	120000	3.9219	0.65	0.6433
0.0373	61.0	122000	3.9035	0.6467	0.6406
0.0354	62.0	124000	3.9745	0.6492	0.6453
0.0412	63.0	126000	3.8600	0.6508	0.6436
0.0347	64.0	128000	3.9549	0.6458	0.6369
0.026	65.0	130000	4.0143	0.6492	0.6455
0.0322	66.0	132000	3.9391	0.6583	0.6518
0.0209	67.0	134000	3.9041	0.6583	0.6480
0.0444	68.0	136000	4.0050	0.6517	0.6471
0.0468	69.0	138000	3.9229	0.6508	0.6433
0.0348	70.0	140000	4.0621	0.6483	0.6423
0.0336	71.0	142000	3.9194	0.6542	0.6478
0.0331	72.0	144000	3.9868	0.6517	0.6394
0.0263	73.0	146000	3.9032	0.6467	0.6380
0.0289	74.0	148000	4.0713	0.6417	0.6342
0.0301	75.0	150000	4.0151	0.6433	0.6341
0.0312	76.0	152000	3.9339	0.6533	0.6452
0.0371	77.0	154000	3.9741	0.6542	0.6463
0.0273	78.0	156000	4.0162	0.6525	0.6433

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1