w2v2-base-pretrained_lr1e-4_at0.8_da0.5

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 12.5971
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 32
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• training_steps: 3500
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
22.3588	10.87	250	23.6457	1.0
8.9908	21.74	500	12.4169	1.0
11.1075	32.61	750	20.5658	1.0
13.3015	43.48	1000	20.3357	1.0
10.7105	54.35	1250	12.4533	1.0
8.8951	65.22	1500	12.5400	1.0
8.8229	76.09	1750	12.5416	1.0
8.7878	86.96	2000	12.5533	1.0
8.8351	97.83	2250	12.5107	1.0
8.8182	108.7	2500	12.6065	1.0
8.7863	119.57	2750	12.6950	1.0
8.801	130.43	3000	12.6095	1.0
8.8417	141.3	3250	12.6030	1.0
8.7401	152.17	3500	12.5971	1.0

Framework versions

• Transformers 4.35.0
• Pytorch 2.0.0
• Datasets 2.14.6
• Tokenizers 0.14.1