File size: 1,414 Bytes
00bcd4e bd97474 a8938d4 816e59e fb27021 5f007ed ab236f3 8cf2dd9 954e6d7 ab236f3 5f007ed fd14a55 2d2cff6 55047be 7d7ea21 f72d954 1c64797 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
---
license: apache-2.0
---
> [!TIP]
> My upload speeds have been cooked and unstable lately. <br>
> Realistically I'd need to move to get a better provider. <br>
> If you **want** and you are able to, you can [**support that endeavor and others here (Ko-fi)**](https://ko-fi.com/Lewdiculous). I apologize for disrupting your experience.
# #llama-3 #experimental #work-in-progress
GGUF-IQ-Imatrix quants for @jeiku's [ResplendentAI/SOVL_Llama3_8B](https://huggingface.co/ResplendentAI/SOVL_Llama3_8B). <br> Give them some love!
> [!IMPORTANT]
> **Updated!**
> These quants have been redone with the fixes from [llama.cpp/pull/6920](https://github.com/ggerganov/llama.cpp/pull/6920) in mind. <br>
> Use **KoboldCpp version 1.64** or higher.
> [!NOTE]
> **Well...!** <br>
> Turns out it was not just a hallucination and this model actually is pretty cool so **give it a chance!** <br>
> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** quant for up to 12288 context sizes.
> [!WARNING]
> **Use the provided presets.** <br>
> Compatible SillyTavern presets [here (simple)](https://huggingface.co/Lewdiculous/Model-Requests/tree/main/data/presets/cope-llama-3-0.1) or [here (Virt's roleplay)](https://huggingface.co/Virt-io/SillyTavern-Presets).
> Use the latest version of KoboldCpp.
 |