w2v2-base-pretrained_lr5e-5_at0.1_da1

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1584
• Wer: 0.1683

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 4000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
19.3795	3.91	250	4.4039	1.0
3.4426	7.81	500	3.2239	1.0
3.1246	11.72	750	3.1005	1.0
2.3409	15.62	1000	1.0214	0.8765
0.6128	19.53	1250	0.6213	0.4643
0.3506	23.44	1500	0.6847	0.2409
0.2498	27.34	1750	0.7219	0.2055
0.1928	31.25	2000	0.9377	0.1730
0.1689	35.16	2250	1.0552	0.1794
0.1437	39.06	2500	0.9440	0.1756
0.1256	42.97	2750	1.1132	0.1721
0.1199	46.88	3000	1.1349	0.1777
0.1087	50.78	3250	1.0542	0.1781
0.1056	54.69	3500	1.1394	0.1696
0.1006	58.59	3750	1.1261	0.1696
0.0933	62.5	4000	1.1584	0.1683

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1