Lewdiculous
commited on
Commit
•
f72d954
1
Parent(s):
8cf2dd9
Update README.md
Browse files
README.md
CHANGED
@@ -8,7 +8,7 @@ GGUF-IQ-Imatrix quants for @jeiku's [ResplendentAI/SOVL_Llama3_8B](https://huggi
|
|
8 |
> [!IMPORTANT]
|
9 |
> **Updated!**
|
10 |
> These quants have been redone with the fixes from [llama.cpp/pull/6920](https://github.com/ggerganov/llama.cpp/pull/6920) in mind. <br>
|
11 |
-
> Use **KoboldCpp version 1.64** or higher.
|
12 |
|
13 |
> [!NOTE]
|
14 |
> **Well...!** <br>
|
@@ -16,7 +16,8 @@ GGUF-IQ-Imatrix quants for @jeiku's [ResplendentAI/SOVL_Llama3_8B](https://huggi
|
|
16 |
> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** quant for up to 12288 context sizes.
|
17 |
|
18 |
> [!WARNING]
|
19 |
-
>
|
20 |
-
>
|
|
|
21 |
|
22 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/626dfb8786671a29c715f8a9/N_1D87adbMuMlSIQ5rI3_.png)
|
|
|
8 |
> [!IMPORTANT]
|
9 |
> **Updated!**
|
10 |
> These quants have been redone with the fixes from [llama.cpp/pull/6920](https://github.com/ggerganov/llama.cpp/pull/6920) in mind. <br>
|
11 |
+
> Use **KoboldCpp version 1.64 (coming soon)** or higher.
|
12 |
|
13 |
> [!NOTE]
|
14 |
> **Well...!** <br>
|
|
|
16 |
> For **8GB VRAM** GPUs, I recommend the **Q4_K_M-imat** quant for up to 12288 context sizes.
|
17 |
|
18 |
> [!WARNING]
|
19 |
+
> **Use the provided presets.** <br>
|
20 |
+
> Compatible SillyTavern presets [here (simple)](https://huggingface.co/Lewdiculous/Model-Requests/tree/main/data/presets/cope-llama-3-0.1) or [here (Virt's roleplay)](https://huggingface.co/Virt-io/SillyTavern-Presets).
|
21 |
+
> Use the latest version of KoboldCpp.
|
22 |
|
23 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/626dfb8786671a29c715f8a9/N_1D87adbMuMlSIQ5rI3_.png)
|