WasuratS/whisper-base-danish-finetune-common-voice-11

Whisper Base Danish - WasuratS

This model is a fine-tuned version of openai/whisper-base on the Common Voice 13 dataset. It achieves the following results on the evaluation set:

• Loss: 0.9795
• Wer Ortho: 45.5986
• Wer: 39.7363

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• training_steps: 6000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Ortho	Wer
0.5156	1.61	500	0.7387	47.8293	42.2586
0.2086	3.22	1000	0.7157	46.7087	41.0652
0.1439	4.82	1500	0.7300	46.5367	40.9610
0.0514	6.43	2000	0.7804	45.2963	39.5279
0.027	8.04	2500	0.8314	46.3126	40.3825
0.0133	9.65	3000	0.8739	44.8585	39.2777
0.0053	11.25	3500	0.9081	45.4839	39.7103
0.0041	12.86	4000	0.9347	45.4110	39.7050
0.0028	14.47	4500	0.9535	46.0624	40.3096
0.0024	16.08	5000	0.9673	45.6351	39.8979
0.0021	17.68	5500	0.9762	45.7862	39.9187
0.002	19.29	6000	0.9795	45.5986	39.7363

Framework versions

• Transformers 4.29.2
• Pytorch 1.13.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3