Llamacpp quants

.gitattributes

@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
+LiberatedHermes-2-Pro-Mistral-7B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text

LiberatedHermes-2-Pro-Mistral-7B-Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef48a716c34c26aef6461379362bb06ee04733f8bb1ea961f6b89621d36fe809
+size 2719393856

LiberatedHermes-2-Pro-Mistral-7B-Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:379aa4f4f7141dff097e9ba55b7f6802448eddd22bed1587d91a5169636f445b
+size 3822189632

LiberatedHermes-2-Pro-Mistral-7B-Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc4955f0a3654251068da6efb56a7bd27d2fd36cf36482afc072658240cb8c6c
+size 3519151168

LiberatedHermes-2-Pro-Mistral-7B-Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7cca0e86962e5928d29e6f6be68bd0001a392dd80617b0993ead8bc6564cef3
+size 3164732480

LiberatedHermes-2-Pro-Mistral-7B-Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15577ba11c955e3c4cd83c1b490e138b5404e89dd17ad431c1f532ba87c1ad09
+size 4109099072

LiberatedHermes-2-Pro-Mistral-7B-Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:930a414e3b895dcdec09a109fed2340ec70707978d29cbea275f26243f2fe50c
+size 4368621632

LiberatedHermes-2-Pro-Mistral-7B-Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a33c66cab6794d481f91f9324da87acac4e9e01be9be44f6f5efb915e0637ba
+size 4140556352

LiberatedHermes-2-Pro-Mistral-7B-Q5_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6cd8bdf24c6f3364facaeca26c9d82fa53d709d3b36e9c68305f8b86f471f667
+size 4997914688

LiberatedHermes-2-Pro-Mistral-7B-Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aa698e507754325a0d94f8712807d67a8cf17243ee752ab3e1f0bca873c26563
+size 5131608128

LiberatedHermes-2-Pro-Mistral-7B-Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a79c17104c518b3209137aabee199466a60b2359bc8b9f586ab40ec413033a89
+size 4997914688

LiberatedHermes-2-Pro-Mistral-7B-Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b892a668dde911cee73c029578ebbae63d92734cd4820f05c9fa3547ba5491fb
+size 5942281280

LiberatedHermes-2-Pro-Mistral-7B-Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a7b52ff8b27ca20733f5d0092e5eeb4aad87b0c9811839b2d655aef0d163fbb
+size 7696137280

README.md

@@ -0,0 +1,32 @@
+---
+license: apache-2.0
+base_model: NousResearch/Hermes-2-Pro-Mistral-7B
+datasets: abacusai/SystemChat
+quantized_by: bartowski
+pipeline_tag: text-generation
+---
+
+## Llamacpp Quantizations of LiberatedHermes-2-Pro-Mistral-7B
+
+Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
+
+Original model: https://huggingface.co/macadeliccc/LiberatedHermes-2-Pro-Mistral-7B
+
+Download a file (not the whole branch) from below:
+
+| Filename | Quant type | File Size | Description |
+| -------- | ---------- | --------- | ----------- |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q8_0.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q6_K.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q5_K_M.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q5_K_S.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q5_0.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q4_K_M.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q4_K_S.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q4_0.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q3_K_L.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q3_K_M.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q3_K_S.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
+| [LiberatedHermes-2-Pro-Mistral-7B-Q2_K.gguf](https://huggingface.co/bartowski/LiberatedHermes-2-Pro-Mistral-7B-GGUF/blob/main/LiberatedHermes-2-Pro-Mistral-7B-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
+
+Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski