MelanieKoe's picture
Model save
569abcb verified
metadata
license: apache-2.0
base_model: facebook/wav2vec2-base
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: w2v2-base-pretrained_lr5e-5_at0.8_da0.05
    results: []

w2v2-base-pretrained_lr5e-5_at0.8_da0.05

This model is a fine-tuned version of facebook/wav2vec2-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 3.9827
  • Wer: 1.0295

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 4000
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
43.0808 83.33 250 28.8440 1.0
8.5391 166.67 500 3.7744 1.0
3.2564 250.0 750 3.1400 1.0
3.0586 333.33 1000 3.1144 1.0
2.9982 416.67 1250 3.0792 1.0
2.927 500.0 1500 3.0777 1.0
2.7925 583.33 1750 3.0732 1.0
2.577 666.67 2000 3.0084 1.0
2.2523 750.0 2250 2.9828 1.0
1.8719 833.33 2500 3.1056 1.0026
1.5177 916.67 2750 3.3320 1.0154
1.233 1000.0 3000 3.5057 1.0184
1.0527 1083.33 3250 3.7065 1.0325
0.9262 1166.67 3500 3.8646 1.0316
0.8456 1250.0 3750 3.9958 1.0295
0.8247 1333.33 4000 3.9827 1.0295

Framework versions

  • Transformers 4.35.0
  • Pytorch 2.0.0
  • Datasets 2.14.6
  • Tokenizers 0.14.1