alexmarques
commited on
Commit
•
33c68d0
1
Parent(s):
8e42fca
Update README.md
Browse files
README.md
CHANGED
@@ -32,7 +32,7 @@ base_model: meta-llama/Meta-Llama-3.1-70B-Instruct
|
|
32 |
- **Model Developers:** Neural Magic
|
33 |
|
34 |
Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
|
35 |
-
It achieves scores within 3.
|
36 |
|
37 |
### Model Optimizations
|
38 |
|
@@ -187,9 +187,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
187 |
</td>
|
188 |
<td>95.38
|
189 |
</td>
|
190 |
-
<td>
|
191 |
</td>
|
192 |
-
<td>
|
193 |
</td>
|
194 |
</tr>
|
195 |
<tr>
|
@@ -197,9 +197,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
197 |
</td>
|
198 |
<td>86.66
|
199 |
</td>
|
200 |
-
<td>86.
|
201 |
</td>
|
202 |
-
<td>99.
|
203 |
</td>
|
204 |
</tr>
|
205 |
<tr>
|
@@ -207,9 +207,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
207 |
</td>
|
208 |
<td>85.32
|
209 |
</td>
|
210 |
-
<td>
|
211 |
</td>
|
212 |
-
<td>
|
213 |
</td>
|
214 |
</tr>
|
215 |
<tr>
|
@@ -217,9 +217,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
217 |
</td>
|
218 |
<td>60.65
|
219 |
</td>
|
220 |
-
<td>
|
221 |
</td>
|
222 |
-
<td>97.
|
223 |
</td>
|
224 |
</tr>
|
225 |
<tr>
|
@@ -227,9 +227,9 @@ This version of the lm-evaluation-harness includes versions of MMLU, ARC-Challen
|
|
227 |
</td>
|
228 |
<td><strong>84.50</strong>
|
229 |
</td>
|
230 |
-
<td><strong>82.
|
231 |
</td>
|
232 |
-
<td><strong>98.
|
233 |
</td>
|
234 |
</tr>
|
235 |
</table>
|
|
|
32 |
- **Model Developers:** Neural Magic
|
33 |
|
34 |
Quantized version of [Meta-Llama-3.1-70B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct).
|
35 |
+
It achieves scores within 3.2% of the scores of the unquantized model for MMLU, ARC-Challenge, GSM-8k, Hellaswag, Winogrande and TruthfulQA.
|
36 |
|
37 |
### Model Optimizations
|
38 |
|
|
|
187 |
</td>
|
188 |
<td>95.38
|
189 |
</td>
|
190 |
+
<td>92.34
|
191 |
</td>
|
192 |
+
<td>96.8%
|
193 |
</td>
|
194 |
</tr>
|
195 |
<tr>
|
|
|
197 |
</td>
|
198 |
<td>86.66
|
199 |
</td>
|
200 |
+
<td>86.01
|
201 |
</td>
|
202 |
+
<td>99.3%
|
203 |
</td>
|
204 |
</tr>
|
205 |
<tr>
|
|
|
207 |
</td>
|
208 |
<td>85.32
|
209 |
</td>
|
210 |
+
<td>85.56
|
211 |
</td>
|
212 |
+
<td>100.3%
|
213 |
</td>
|
214 |
</tr>
|
215 |
<tr>
|
|
|
217 |
</td>
|
218 |
<td>60.65
|
219 |
</td>
|
220 |
+
<td>59.39
|
221 |
</td>
|
222 |
+
<td>97.9%
|
223 |
</td>
|
224 |
</tr>
|
225 |
<tr>
|
|
|
227 |
</td>
|
228 |
<td><strong>84.50</strong>
|
229 |
</td>
|
230 |
+
<td><strong>82.98</strong>
|
231 |
</td>
|
232 |
+
<td><strong>98.2%</strong>
|
233 |
</td>
|
234 |
</tr>
|
235 |
</table>
|