tun_wav2vec5

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2565
• Wer: 0.6445
• Cer: 0.1660

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 10
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
1.7385	5.5556	200	1.1520	0.9062	0.3122
0.9367	11.1111	400	1.0713	0.8594	0.2677
0.5837	16.6667	600	1.0317	0.7656	0.2218
0.4166	22.2222	800	1.0374	0.8008	0.2218
0.3707	27.7778	1000	1.0956	0.7734	0.2157
0.2989	33.3333	1200	1.1494	0.7734	0.2096
0.2404	38.8889	1400	1.3076	0.7383	0.1960
0.2142	44.4444	1600	1.1622	0.7031	0.1852
0.1709	50.0	1800	1.1106	0.7227	0.1903
0.1599	55.5556	2000	1.3641	0.7109	0.1894
0.1464	61.1111	2200	1.3258	0.6992	0.1739
0.1283	66.6667	2400	1.1508	0.6680	0.1753
0.1485	72.2222	2600	1.3598	0.6484	0.1744
0.11	77.7778	2800	1.4104	0.6758	0.1810
0.0967	83.3333	3000	1.3813	0.6797	0.1782
0.0844	88.8889	3200	1.1982	0.6523	0.1697
0.0876	94.4444	3400	1.2400	0.6484	0.1711
0.0784	100.0	3600	1.2565	0.6445	0.1660

Framework versions

• Transformers 4.42.3
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1