xls-r-1b-bem-genbed-m-model

This model is a fine-tuned version of facebook/wav2vec2-xls-r-1b on the GENBED - BEM dataset. It achieves the following results on the evaluation set:

• Loss: 0.5624
• Wer: 0.7514

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30.0
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
No log	0.2759	100	2.9022	1.0
No log	0.5517	200	1.4915	1.0022
No log	0.8276	300	0.9387	0.9416
No log	1.1034	400	0.7254	0.8661
2.1327	1.3793	500	0.8557	0.9420
2.1327	1.6552	600	0.7236	0.8662
2.1327	1.9310	700	0.6982	0.8573
2.1327	2.2069	800	0.6066	0.8003
2.1327	2.4828	900	0.6352	0.7989
0.733	2.7586	1000	0.5855	0.7902
0.733	3.0345	1100	0.5587	0.7390
0.733	3.3103	1200	0.5624	0.7514
0.733	3.5862	1300	0.5214	0.7193
0.733	3.8621	1400	0.5321	0.7208
0.5894	4.1379	1500	0.6477	0.7799
0.5894	4.4138	1600	0.6076	0.7810
0.5894	4.6897	1700	0.5821	0.7666

Framework versions

• Transformers 4.46.0.dev0
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0