---
license: apache-2.0
base_model: Helsinki/opus-mt-en-mul
datasets:
- ai4bharat/samanantar
- wmt/wmt19
language:
- en
- ta
- gu
metrics:
- bleu
---

# Finetuning

This model is a fine-tuned version of [Varsha00/finetuned-opusmt-en-to-ta](https://huggingface.co/Varsha00/finetuned-opusmt-en-to-ta) on the samanantar & WMT News dataset.
source group: English
target group: Gujarati
model: transformer

## Model description

This model is a sequentially finetuned version of the Helsinki-NLP/opus-mt-en-mul model, designed for translating between English and Gujarati. 
The model was initially finetuned on the Tamil language using a substantial dataset and subsequently finetuned on Gujarati using a smaller dataset. 
This approach, known as sequential finetuning or cascaded finetuning, allows the model to leverage the knowledge gained from Hindi to improve its 
performance on Gujarati translations, despite the limited data available for the latter.

## Training and evaluation data

ai4bharath/samanantar
WMT-News 

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-5
- warmup_steps: 500
- gradient_accumulation_steps: 2
- total_train_batch_size: 16
- num_epochs: 10

### Benchamark Evaluation
- BLEU score on Tatoeba: 26.2690989442415
- BLUE score on IN-22: 5.81105590832792

### Framework versions

- Transformers 4.42.3
- Pytorch 2.1.2
- Datasets 2.20.0
- Tokenizers 0.19.1