alexmarques commited on
Commit
33c68d0
1 Parent(s): 8e42fca

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -11
README.md CHANGED
@@ -32,7 +32,7 @@ base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
32
  - **Model Developers:** Neural Magic
33
 
34
  Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
35
- It achieves scores within 3.1% of the scores of the unquantized model for MMLU, ARC-Challenge, GSM-8k, Hellaswag, Winogrande and TruthfulQA.
36
 
37
  ### Model Optimizations
38
 
@@ -187,9 +187,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
187
  </td>
188
  <td>95.38
189
  </td>
190
- <td>93.25
191
  </td>
192
- <td>97.8%
193
  </td>
194
  </tr>
195
  <tr>
@@ -197,9 +197,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
197
  </td>
198
  <td>86.66
199
  </td>
200
- <td>86.16
201
  </td>
202
- <td>99.4%
203
  </td>
204
  </tr>
205
  <tr>
@@ -207,9 +207,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
207
  </td>
208
  <td>85.32
209
  </td>
210
- <td>84.14
211
  </td>
212
- <td>98.6%
213
  </td>
214
  </tr>
215
  <tr>
@@ -217,9 +217,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
217
  </td>
218
  <td>60.65
219
  </td>
220
- <td>58.89
221
  </td>
222
- <td>97.1%
223
  </td>
224
  </tr>
225
  <tr>
@@ -227,9 +227,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
227
  </td>
228
  <td><strong>84.50</strong>
229
  </td>
230
- <td><strong>82.85</strong>
231
  </td>
232
- <td><strong>98.0%</strong>
233
  </td>
234
  </tr>
235
  </table>
 
32
  - **Model Developers:** Neural Magic
33
 
34
  Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
35
+ It achieves scores within 3.2% of the scores of the unquantized model for MMLU, ARC-Challenge, GSM-8k, Hellaswag, Winogrande and TruthfulQA.
36
 
37
  ### Model Optimizations
38
 
 
187
  </td>
188
  <td>95.38
189
  </td>
190
+ <td>92.34
191
  </td>
192
+ <td>96.8%
193
  </td>
194
  </tr>
195
  <tr>
 
197
  </td>
198
  <td>86.66
199
  </td>
200
+ <td>86.01
201
  </td>
202
+ <td>99.3%
203
  </td>
204
  </tr>
205
  <tr>
 
207
  </td>
208
  <td>85.32
209
  </td>
210
+ <td>85.56
211
  </td>
212
+ <td>100.3%
213
  </td>
214
  </tr>
215
  <tr>
 
217
  </td>
218
  <td>60.65
219
  </td>
220
+ <td>59.39
221
  </td>
222
+ <td>97.9%
223
  </td>
224
  </tr>
225
  <tr>
 
227
  </td>
228
  <td><strong>84.50</strong>
229
  </td>
230
+ <td><strong>82.98</strong>
231
  </td>
232
+ <td><strong>98.2%</strong>
233
  </td>
234
  </tr>
235
  </table>