Hugging Face's logo

bartowski
/
zephyr-7b-spin-iter3-v0-GGUF

Text Generation
GGUF
Generated from Trainer
Inference Endpoints
conversational
Model card Files Community
bartowski commited on
Commit
a613038
•
1 Parent(s): d25f6f3

Llamacpp quants

Browse files
Files changed (14) hide show
  1. .gitattributes +12 -0
  2. README.md +40 -0
  3. zephyr-7b-spin-iter3-v0-Q2_K.gguf +3 -0
  4. zephyr-7b-spin-iter3-v0-Q3_K_L.gguf +3 -0
  5. zephyr-7b-spin-iter3-v0-Q3_K_M.gguf +3 -0
  6. zephyr-7b-spin-iter3-v0-Q3_K_S.gguf +3 -0
  7. zephyr-7b-spin-iter3-v0-Q4_0.gguf +3 -0
  8. zephyr-7b-spin-iter3-v0-Q4_K_M.gguf +3 -0
  9. zephyr-7b-spin-iter3-v0-Q4_K_S.gguf +3 -0
  10. zephyr-7b-spin-iter3-v0-Q5_0.gguf +3 -0
  11. zephyr-7b-spin-iter3-v0-Q5_K_M.gguf +3 -0
  12. zephyr-7b-spin-iter3-v0-Q5_K_S.gguf +3 -0
  13. zephyr-7b-spin-iter3-v0-Q6_K.gguf +3 -0
  14. zephyr-7b-spin-iter3-v0-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,15 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ zephyr-7b-spin-iter3-v0-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
37
+ zephyr-7b-spin-iter3-v0-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
38
+ zephyr-7b-spin-iter3-v0-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
39
+ zephyr-7b-spin-iter3-v0-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
40
+ zephyr-7b-spin-iter3-v0-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
41
+ zephyr-7b-spin-iter3-v0-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ zephyr-7b-spin-iter3-v0-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ zephyr-7b-spin-iter3-v0-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
44
+ zephyr-7b-spin-iter3-v0-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
45
+ zephyr-7b-spin-iter3-v0-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
46
+ zephyr-7b-spin-iter3-v0-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
47
+ zephyr-7b-spin-iter3-v0-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: argilla/zephyr-7b-spin-iter2-v0
4
+ tags:
5
+ - generated_from_trainer
6
+ model-index:
7
+ - name: zephyr-7b-spin-iter3-v0
8
+ results: []
9
+ datasets:
10
+ - argilla/10k_prompts_SPIN_iter3_zephyr_top
11
+ - argilla/10k_prompts_SPIN_iter2_zephyr_top
12
+ - DIBT/10k_prompts_ranked
13
+ quantized_by: bartowski
14
+ pipeline_tag: text-generation
15
+ ---
16
+
17
+ ## Llamacpp Quantizations of zephyr-7b-spin-iter3-v0
18
+
19
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
20
+
21
+ Original model: https://huggingface.co/argilla/zephyr-7b-spin-iter3-v0
22
+
23
+ Download a file (not the whole branch) from below:
24
+
25
+ | Filename | Quant type | File Size | Description |
26
+ | -------- | ---------- | --------- | ----------- |
27
+ | [zephyr-7b-spin-iter3-v0-Q8_0.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q8_0.gguf) | Q8_0 | 7.69GB | Extremely high quality, generally unneeded but max available quant. |
28
+ | [zephyr-7b-spin-iter3-v0-Q6_K.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q6_K.gguf) | Q6_K | 5.94GB | Very high quality, near perfect, *recommended*. |
29
+ | [zephyr-7b-spin-iter3-v0-Q5_K_M.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q5_K_M.gguf) | Q5_K_M | 5.13GB | High quality, very usable. |
30
+ | [zephyr-7b-spin-iter3-v0-Q5_K_S.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q5_K_S.gguf) | Q5_K_S | 4.99GB | High quality, very usable. |
31
+ | [zephyr-7b-spin-iter3-v0-Q5_0.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q5_0.gguf) | Q5_0 | 4.99GB | High quality, older format, generally not recommended. |
32
+ | [zephyr-7b-spin-iter3-v0-Q4_K_M.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q4_K_M.gguf) | Q4_K_M | 4.36GB | Good quality, similar to 4.25 bpw. |
33
+ | [zephyr-7b-spin-iter3-v0-Q4_K_S.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q4_K_S.gguf) | Q4_K_S | 4.14GB | Slightly lower quality with small space savings. |
34
+ | [zephyr-7b-spin-iter3-v0-Q4_0.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q4_0.gguf) | Q4_0 | 4.10GB | Decent quality, older format, generally not recommended. |
35
+ | [zephyr-7b-spin-iter3-v0-Q3_K_L.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q3_K_L.gguf) | Q3_K_L | 3.82GB | Lower quality but usable, good for low RAM availability. |
36
+ | [zephyr-7b-spin-iter3-v0-Q3_K_M.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q3_K_M.gguf) | Q3_K_M | 3.51GB | Even lower quality. |
37
+ | [zephyr-7b-spin-iter3-v0-Q3_K_S.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q3_K_S.gguf) | Q3_K_S | 3.16GB | Low quality, not recommended. |
38
+ | [zephyr-7b-spin-iter3-v0-Q2_K.gguf](https://huggingface.co/bartowski/zephyr-7b-spin-iter3-v0-GGUF/blob/main/zephyr-7b-spin-iter3-v0-Q2_K.gguf) | Q2_K | 2.71GB | Extremely low quality, *not* recommended.
39
+
40
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
zephyr-7b-spin-iter3-v0-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:990bb269c0da8241da2c68dd126b8ebdb79e0e9a61c46be0104de8672f64c51a
3
+ size 2719242464
zephyr-7b-spin-iter3-v0-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ffd6d9ce054e6fefb253eeb51b7a5ed5f1fedb5e478cecda29426c49ec083877
3
+ size 3822024928
zephyr-7b-spin-iter3-v0-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4206cbdcf889ee2512d13eff9ac8f9373cdca108a7126fb0da2a26d6e7e8314d
3
+ size 3518986464
zephyr-7b-spin-iter3-v0-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e3c810a8ffe1e3e71322035f98a2e1888de55dd3a395671762fdea9607a4d06e
3
+ size 3164567776
zephyr-7b-spin-iter3-v0-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8ab3283a0537c8e0cdd8c16fec50e04058d44dc8338aaaa0e19bdf2e11df25a
3
+ size 4108916960
zephyr-7b-spin-iter3-v0-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:139a8497211fe4cd9ad617313eb20915686eb7d4f37f2d006503c1b0edd9fd22
3
+ size 4368439520
zephyr-7b-spin-iter3-v0-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3cfffcc4c114d461d1a97798bf4776263b63c2fc86d98c368ace7105fe4fcd3
3
+ size 4140374240
zephyr-7b-spin-iter3-v0-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f7182f9ea0a05bd5fd8fe63460b65d05723646bb0f1cf9fd29b5a5f5224cb32
3
+ size 4997716192
zephyr-7b-spin-iter3-v0-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:cac5ca81424fd74063541582598a9855467dea3cb132cb4fb0223ae6811b238b
3
+ size 5131409632
zephyr-7b-spin-iter3-v0-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec89927076d46b96ca9ff4fde0295ed35570c2eb2dc64270041a94212f18e51c
3
+ size 4997716192
zephyr-7b-spin-iter3-v0-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:90f13f0e13e29ea96d013f7c38fc4ffbd072adceb2bebb26abf22d7f5fccbca2
3
+ size 5942065376
zephyr-7b-spin-iter3-v0-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f7488f87851ca0aac4916abb4dd228893102422e7c7f0805e1bbdfd476cfb32
3
+ size 7695857888