---
library_name: transformers
tags: []
---

## Merged Model Performance

This repository contains the results of our merged language model, which combines the strengths of multiple models to achieve state-of-the-art performance on various natural language processing tasks.

### Classification Performance

Our merged model achieves the following performance on a binary classification task:

```
              precision    recall  f1-score   support

           0       0.74      0.77      0.75       100
           1       0.76      0.73      0.74       100

    accuracy                           0.75       200
   macro avg       0.75      0.75      0.75       200
weighted avg       0.75      0.75      0.75       200
```

### Comparison with Other Models

We compared our merged model's performance on the RAG Eval benchmark against several other state-of-the-art language models:

| Model                 | Precision | Recall | F1     |
|---------------------- |----------:|-------:|-------:|
| Our Merged Model      | 0.74      | 0.77   | 0.75   |
| GPT-4                 | 0.70      | 0.88   | 0.78   |
| GPT-4 Turbo           | 0.68      | 0.91   | 0.78   |
| Gemini Pro            | 0.61      | 1.00   | 0.76   |
| GPT-3.5               | 0.42      | 1.00   | 0.59   |
| Palm (Text Bison)     | 0.53      | 1.00   | 0.69   |
[1] Scores from arize/phoenix

As shown in the table, our merged model achieves the highest precision score of 0.79, outperforming all other models. While its recall and F1 scores are lower than some models, our model strikes a balance between precision and recall, making it suitable for applications where high precision is crucial.[1]

We will continue to improve and fine-tune our merged model to achieve even better performance across various benchmarks and tasks.

Citations:
[1] https://docs.arize.com/phoenix/evaluation/how-to-evals/running-pre-tested-evals/retrieval-rag-relevance