InferenceIllusionist
commited on
Commit
•
33fcd11
1
Parent(s):
17ec74d
Update README.md
Browse files
README.md
CHANGED
@@ -7,12 +7,24 @@ tags:
|
|
7 |
- gguf
|
8 |
- iMat
|
9 |
---
|
|
|
|
|
10 |
# DarkForest 20B v2.0 iMat GGUF
|
|
|
11 |
<h4><i>"The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him."- Liu Cixin</i></h4>
|
12 |
|
13 |
|
14 |
-
Importance Matrix calculated using Q8_0 quant and wiki.train.raw
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
15 |
|
|
|
16 |
|
17 |
<details><summary>Previous Model Card</summary>
|
18 |
Continuation of an ongoing initiative to bring the latest and greatest models to consumer hardware through SOTA techniques that reduce VRAM overhead.
|
|
|
7 |
- gguf
|
8 |
- iMat
|
9 |
---
|
10 |
+
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
|
11 |
+
|
12 |
# DarkForest 20B v2.0 iMat GGUF
|
13 |
+
|
14 |
<h4><i>"The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him."- Liu Cixin</i></h4>
|
15 |
|
16 |
|
17 |
+
Quantized from fp16 with love. Importance Matrix calculated using Q8_0 quant and wiki.train.raw
|
18 |
+
|
19 |
+
For a brief rundown of iMatrix quant performance please see this [PR](https://github.com/ggerganov/llama.cpp/pull/5747)
|
20 |
+
|
21 |
+
<i>All quants are verified working prior to uploading to repo for your safety and convenience. </i>
|
22 |
+
|
23 |
+
Importance matrix quantizations are a work in progress, IQ3 and above is recommended for best results.
|
24 |
+
|
25 |
+
<b>Tip:</b> Pick a size that can fit in your GPU while still allowing some room for context for best speed. You may need to pad this further depending on if you are running image gen or TTS as well.
|
26 |
|
27 |
+
Original model card can be found [here](https://huggingface.co/TeeZee/DarkForest-20B-v2.0)
|
28 |
|
29 |
<details><summary>Previous Model Card</summary>
|
30 |
Continuation of an ongoing initiative to bring the latest and greatest models to consumer hardware through SOTA techniques that reduce VRAM overhead.
|