InferenceIllusionist
commited on
Commit
•
57ef77b
1
Parent(s):
5e73dfc
Update README.md
Browse files
README.md
CHANGED
@@ -11,6 +11,10 @@ tags:
|
|
11 |
<h4><i>"The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him."- Liu Cixin</i></h4>
|
12 |
|
13 |
|
|
|
|
|
|
|
|
|
14 |
Continuation of an ongoing initiative to bring the latest and greatest models to consumer hardware through SOTA techniques that reduce VRAM overhead.
|
15 |
|
16 |
After testing the new important matrix quants for 11b and 8x7b models and being able to run them on machines without a dedicated GPU, we are now exploring the middleground - 20b.
|
@@ -23,6 +27,8 @@ IQ3_S has been generated after PR [#5829](https://github.com/ggerganov/llama.cpp
|
|
23 |
|
24 |
(Credits to [TeeZee](https://huggingface.co/TeeZee/) for the original model and [ikawrakow](https://github.com/ikawrakow) for the stellar work on IQ quants)
|
25 |
|
|
|
|
|
26 |
---
|
27 |
|
28 |
# DarkForest 20B v2.0
|
|
|
11 |
<h4><i>"The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him."- Liu Cixin</i></h4>
|
12 |
|
13 |
|
14 |
+
Importance Matrix calculated using Q8_0 quant and wiki.train.raw
|
15 |
+
|
16 |
+
|
17 |
+
<details><summary>Previous Model Card</summary>
|
18 |
Continuation of an ongoing initiative to bring the latest and greatest models to consumer hardware through SOTA techniques that reduce VRAM overhead.
|
19 |
|
20 |
After testing the new important matrix quants for 11b and 8x7b models and being able to run them on machines without a dedicated GPU, we are now exploring the middleground - 20b.
|
|
|
27 |
|
28 |
(Credits to [TeeZee](https://huggingface.co/TeeZee/) for the original model and [ikawrakow](https://github.com/ikawrakow) for the stellar work on IQ quants)
|
29 |
|
30 |
+
</details><br>
|
31 |
+
|
32 |
---
|
33 |
|
34 |
# DarkForest 20B v2.0
|