Lewdiculous
/

SOVL_Llama3_8B-GGUF-IQ-Imatrix

Inference Endpoints

Model card Files Files and versions Community

Lewdiculous commited on Apr 25

Commit

55047be

•

1 Parent(s): 7a9f9c8

Update README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -8,6 +8,7 @@ license: apache-2.0
 > [!NOTE]
 > **Well...!** <br>
 > Turns out it was not just a hallucination and this model actually is pretty cool so **give it a chance!** <br>
 > [!WARNING]
 > Compatible SillyTavern presets [here (simple)](https://huggingface.co/Lewdiculous/Model-Requests/tree/main/data/presets/cope-llama-3-0.1) or [here (Virt's)](https://huggingface.co/Virt-io/SillyTavern-Presets). <br>

 > [!NOTE]
 > **Well...!** <br>
 > Turns out it was not just a hallucination and this model actually is pretty cool so **give it a chance!** <br>
+> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** quant for up to 12288 context sizes.
 > [!WARNING]
 > Compatible SillyTavern presets [here (simple)](https://huggingface.co/Lewdiculous/Model-Requests/tree/main/data/presets/cope-llama-3-0.1) or [here (Virt's)](https://huggingface.co/Virt-io/SillyTavern-Presets). <br>