InferenceIllusionist commited on
Commit
3d1b231
1 Parent(s): 3894956

Update README.md

Browse files

Adding link & title to graph, tips

Files changed (1) hide show
  1. README.md +5 -3
README.md CHANGED
@@ -19,11 +19,13 @@ Quantized from fp16.
19
 
20
  For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
21
 
22
- <i>All quants are verified working prior to uploading to repo for your safety and convenience. </i>
23
 
24
- <img src="https://i.imgur.com/mV0nYdA.png" width="920"/>
 
 
25
 
26
 
27
- <b>Tip:</b> Pick a file size under your GPU's VRAM while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
28
 
29
  Original model card can be found [here](https://huggingface.co/mistralai/mathstral-7B-v0.1)
 
19
 
20
  For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
21
 
22
+ <i>All quants are verified working prior to uploading to repo for your safety and convenience</i>
23
 
24
+ <b>KL-Divergence Reference Chart</b>
25
+ (Click on image to view in full size)
26
+ [<img src="https://i.imgur.com/mV0nYdA.png" width="920"/>](https://i.imgur.com/mV0nYdA.png)
27
 
28
 
29
+ <b>Tips:</b> There's no need to download the entire repo, just pick one of the GGUF files. As with smaller 7b models, Q6 or larger is recommended for best results. On quants smaller than Q3, repetition penalty = 1.05 - 1.3 and min P = 0.05 mitigated some issues, but set your expectations accordingly
30
 
31
  Original model card can be found [here](https://huggingface.co/mistralai/mathstral-7B-v0.1)