Commit
•
4045143
1
Parent(s):
76b9b85
Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,37 @@ All quants were made using the imatrix option and Bartowski's [calibration file]
|
|
| 25 |
|
| 26 |
# Perplexity table (the lower the better)
|
| 27 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 28 |
<hr>
|
| 29 |
|
| 30 |
# Qwen2.5-32B-Instruct
|
|
|
|
| 25 |
|
| 26 |
# Perplexity table (the lower the better)
|
| 27 |
|
| 28 |
+
| Quant | Size (MB) | PPL | Size (%) | Accuracy (%) | PPL error rate |
|
| 29 |
+
| ------- | --------- | ------- | -------- | ------------ | -------------- |
|
| 30 |
+
| IQ1_S | 6938 | 12.2991 | 11.1 | 44.87 | 0.08384 |
|
| 31 |
+
| IQ1_M | 7565 | 10.2638 | 12.1 | 53.77 | 0.0699 |
|
| 32 |
+
| IQ2_XS | 9497 | 7.3601 | 15.2 | 74.99 | 0.04846 |
|
| 33 |
+
| IQ2_S | 9907 | 7.2397 | 15.85 | 76.23 | 0.04762 |
|
| 34 |
+
| IQ2_M | 10743 | 6.7268 | 17.19 | 82.05 | 0.04354 |
|
| 35 |
+
| Q2_K_S | 10956 | 6.9981 | 17.53 | 78.87 | 0.04644 |
|
| 36 |
+
| Q2_K | 11743 | 6.6603 | 18.79 | 82.87 | 0.04324 |
|
| 37 |
+
| IQ3_XXS | 12245 | 6.157 | 19.59 | 89.64 | 0.03929 |
|
| 38 |
+
| IQ3_XS | 13071 | 6.0366 | 20.91 | 91.43 | 0.03833 |
|
| 39 |
+
| Q3_K_S | 13726 | 6.0878 | 21.96 | 90.66 | 0.03872 |
|
| 40 |
+
| IQ3_S | 13769 | 5.9886 | 22.03 | 92.16 | 0.03816 |
|
| 41 |
+
| IQ3_M | 14125 | 5.9942 | 22.6 | 92.07 | 0.03802 |
|
| 42 |
+
| Q3_K_M | 15197 | 5.8008 | 24.32 | 95.14 | 0.03677 |
|
| 43 |
+
| Q3_K_L | 16449 | 5.7812 | 26.32 | 95.47 | 0.03667 |
|
| 44 |
+
| IQ4_XS | 16874 | 5.6502 | 27 | 97.68 | 0.03586 |
|
| 45 |
+
| IQ4_NL | 17817 | 5.6408 | 28.51 | 97.84 | 0.03575 |
|
| 46 |
+
| Q4_0 | 17845 | 5.6946 | 28.55 | 96.92 | 0.03599 |
|
| 47 |
+
| Q4_K_S | 17915 | 5.6367 | 28.66 | 97.91 | 0.03561 |
|
| 48 |
+
| Q4_K_M | 18932 | 5.6224 | 30.29 | 98.16 | 0.03554 |
|
| 49 |
+
| IQ2_XXS | 8611 | 8.0187 | 13.78 | 68.83 | 0.05388 |
|
| 50 |
+
| Q4_1 | 19684 | 5.6586 | 31.49 | 97.53 | 0.03587 |
|
| 51 |
+
| Q5_K_S | 21590 | 5.568 | 34.54 | 99.12 | 0.03515 |
|
| 52 |
+
| Q5_0 | 21658 | 5.588 | 34.65 | 98.77 | 0.03538 |
|
| 53 |
+
| Q5_K_M | 22185 | 5.567 | 35.5 | 99.14 | 0.03515 |
|
| 54 |
+
| Q5_1 | 23496 | 5.5734 | 37.59 | 99.03 | 0.0352 |
|
| 55 |
+
| Q6_K | 25641 | 5.5305 | 41.03 | 99.79 | 0.03483 |
|
| 56 |
+
| Q8_0 | 33208 | 5.5221 | 53.13 | 99.95 | 0.03478 |
|
| 57 |
+
| F16 | 62500 | 5.5191 | 100 | 100 | 0.03474 |
|
| 58 |
+
|
| 59 |
<hr>
|
| 60 |
|
| 61 |
# Qwen2.5-32B-Instruct
|