wav2vec2-base

This model is a fine-tuned version of facebook/wav2vec2-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5735
• Accuracy: 0.8913

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	3	2.7459	0.1304
No log	1.85	6	2.6837	0.1087
No log	2.77	9	2.6583	0.1087
2.6599	4.0	13	2.6553	0.1087
2.6599	4.92	16	2.5628	0.1522
2.6599	5.85	19	2.4286	0.1739
2.3457	6.77	22	2.4705	0.1522
2.3457	8.0	26	2.2801	0.1522
2.3457	8.92	29	2.2110	0.2391
2.1136	9.85	32	2.1101	0.2391
2.1136	10.77	35	2.0434	0.3478
2.1136	12.0	39	2.2015	0.2609
1.8271	12.92	42	1.8463	0.2826
1.8271	13.85	45	1.8144	0.2391
1.8271	14.77	48	1.6712	0.2391
1.6517	16.0	52	1.6885	0.4348
1.6517	16.92	55	1.7268	0.4565
1.6517	17.85	58	1.5564	0.5435
1.5123	18.77	61	1.4261	0.5435
1.5123	20.0	65	1.2945	0.6739
1.5123	20.92	68	1.2329	0.6957
1.2441	21.85	71	1.1841	0.6957
1.2441	22.77	74	1.1297	0.7174
1.2441	24.0	78	1.0477	0.7826
1.0647	24.92	81	1.0039	0.7174
1.0647	25.85	84	0.9795	0.7174
1.0647	26.77	87	0.9619	0.7609
0.9374	28.0	91	0.8940	0.8043
0.9374	28.92	94	0.8675	0.8043
0.9374	29.85	97	0.8516	0.8043
0.7902	30.77	100	0.8203	0.8261
0.7902	32.0	104	0.7963	0.7609
0.7902	32.92	107	0.7329	0.8478
0.6959	33.85	110	0.7382	0.8043
0.6959	34.77	113	0.7205	0.8261
0.6959	36.0	117	0.6996	0.8043
0.6694	36.92	120	0.6949	0.8696
0.6694	37.85	123	0.7009	0.7826
0.6694	38.77	126	0.6502	0.8261
0.6226	40.0	130	0.5835	0.8478
0.6226	40.92	133	0.5735	0.8913
0.6226	41.85	136	0.5651	0.8913
0.6226	42.77	139	0.5624	0.8913
0.5746	44.0	143	0.5565	0.8913
0.5746	44.92	146	0.5476	0.8913
0.5746	45.85	149	0.5439	0.8913
0.5238	46.15	150	0.5435	0.8913

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1