Adding the Open Portuguese LLM Leaderboard Evaluation Results

#4
Files changed (1) hide show
  1. README.md +25 -6
README.md CHANGED
@@ -1,5 +1,8 @@
1
  ---
2
- base_model: mistralai/Mistral-7B-v0.1
 
 
 
3
  tags:
4
  - mistral
5
  - instruct
@@ -8,13 +11,10 @@ tags:
8
  - gpt4
9
  - synthetic data
10
  - distillation
 
11
  model-index:
12
  - name: OpenHermes-2-Mistral-7B-PTBR
13
  results: []
14
- license: apache-2.0
15
- language:
16
- - en
17
- - pt
18
  ---
19
 
20
  # OpenHermes 2 - Mistral 7B - PT BR
@@ -103,4 +103,23 @@ To utilize the prompt format without a system prompt, simply leave the line out.
103
  Currently, I recommend using LM Studio for chatting with Hermes 2. It is a GUI application that utilizes GGUF models with a llama.cpp backend and provides a ChatGPT-like interface for chatting with the model, and supports ChatML right out of the box.
104
  In LM-Studio, simply select the ChatML Prefix on the settings side pane:
105
 
106
- ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - pt
5
+ license: apache-2.0
6
  tags:
7
  - mistral
8
  - instruct
 
11
  - gpt4
12
  - synthetic data
13
  - distillation
14
+ base_model: mistralai/Mistral-7B-v0.1
15
  model-index:
16
  - name: OpenHermes-2-Mistral-7B-PTBR
17
  results: []
 
 
 
 
18
  ---
19
 
20
  # OpenHermes 2 - Mistral 7B - PT BR
 
103
  Currently, I recommend using LM Studio for chatting with Hermes 2. It is a GUI application that utilizes GGUF models with a llama.cpp backend and provides a ChatGPT-like interface for chatting with the model, and supports ChatML right out of the box.
104
  In LM-Studio, simply select the ChatML Prefix on the settings side pane:
105
 
106
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/6317aade83d8d2fd903192d9/ls6WqV-GSxMw2RA3GuQiN.png)
107
+
108
+
109
+ # Open Portuguese LLM Leaderboard Evaluation Results
110
+
111
+ Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/artificialguybr/OpenHermesV2-PTBR) and on the [πŸš€ Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
112
+
113
+ | Metric | Value |
114
+ |--------------------------|---------|
115
+ |Average |**61.94**|
116
+ |ENEM Challenge (No Images)| 59.90|
117
+ |BLUEX (No Images) | 48.40|
118
+ |OAB Exams | 41.41|
119
+ |Assin2 RTE | 88.89|
120
+ |Assin2 STS | 67.61|
121
+ |FaQuAD NLI | 53.06|
122
+ |HateBR Binary | 74.77|
123
+ |PT Hate Speech Binary | 64.38|
124
+ |tweetSentBR | 59.01|
125
+