w2v2-base-pretrained_lr5e-5_at0.8_da0.8

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2369
• Wer: 0.1717

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
18.8283	6.76	250	4.1824	1.0
3.3429	13.51	500	3.2478	1.0
3.1004	20.27	750	3.1282	1.0
1.8593	27.03	1000	1.1074	0.4259
0.2428	33.78	1250	1.2176	0.2166
0.1279	40.54	1500	1.6077	0.1931
0.0893	47.3	1750	1.7647	0.1867
0.0669	54.05	2000	1.9817	0.1815
0.0527	60.81	2250	2.0652	0.1803
0.0458	67.57	2500	2.1288	0.1850
0.0408	74.32	2750	1.9809	0.1786
0.0345	81.08	3000	2.0693	0.1833
0.031	87.84	3250	2.3649	0.1798
0.0277	94.59	3500	2.3239	0.1721
0.0279	101.35	3750	2.1769	0.1730
0.0263	108.11	4000	2.2369	0.1717

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1