Hugging Face's logo

bartowski
/
blossom-v5-9b-GGUF

Text Generation
GGUF
Chinese
English
Inference Endpoints
Model card Files Community
bartowski commited on
Commit
b11e6e1
•
1 Parent(s): e616e96

Llamacpp quants

Browse files
Files changed (18) hide show
  1. .gitattributes +16 -0
  2. README.md +42 -0
  3. blossom-v5-9b-IQ3_M.gguf +3 -0
  4. blossom-v5-9b-IQ3_S.gguf +3 -0
  5. blossom-v5-9b-IQ4_NL.gguf +3 -0
  6. blossom-v5-9b-IQ4_XS.gguf +3 -0
  7. blossom-v5-9b-Q2_K.gguf +3 -0
  8. blossom-v5-9b-Q3_K_L.gguf +3 -0
  9. blossom-v5-9b-Q3_K_M.gguf +3 -0
  10. blossom-v5-9b-Q3_K_S.gguf +3 -0
  11. blossom-v5-9b-Q4_0.gguf +3 -0
  12. blossom-v5-9b-Q4_K_M.gguf +3 -0
  13. blossom-v5-9b-Q4_K_S.gguf +3 -0
  14. blossom-v5-9b-Q5_0.gguf +3 -0
  15. blossom-v5-9b-Q5_K_M.gguf +3 -0
  16. blossom-v5-9b-Q5_K_S.gguf +3 -0
  17. blossom-v5-9b-Q6_K.gguf +3 -0
  18. blossom-v5-9b-Q8_0.gguf +3 -0
.gitattributes CHANGED
@@ -33,3 +33,19 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ blossom-v5-9b-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
37
+ blossom-v5-9b-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
38
+ blossom-v5-9b-IQ4_NL.gguf filter=lfs diff=lfs merge=lfs -text
39
+ blossom-v5-9b-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
40
+ blossom-v5-9b-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
41
+ blossom-v5-9b-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
42
+ blossom-v5-9b-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
43
+ blossom-v5-9b-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
44
+ blossom-v5-9b-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
45
+ blossom-v5-9b-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
46
+ blossom-v5-9b-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
47
+ blossom-v5-9b-Q5_0.gguf filter=lfs diff=lfs merge=lfs -text
48
+ blossom-v5-9b-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
49
+ blossom-v5-9b-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
50
+ blossom-v5-9b-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
51
+ blossom-v5-9b-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Azure99/blossom-chat-v3
5
+ - Azure99/blossom-math-v4
6
+ - Azure99/blossom-wizard-v3
7
+ - Azure99/blossom-orca-v3
8
+ language:
9
+ - zh
10
+ - en
11
+ quantized_by: bartowski
12
+ pipeline_tag: text-generation
13
+ ---
14
+
15
+ ## Llamacpp Quantizations of blossom-v5-9b
16
+
17
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b2440">b2440</a> for quantization.
18
+
19
+ Original model: https://huggingface.co/Azure99/blossom-v5-9b
20
+
21
+ Download a file (not the whole branch) from below:
22
+
23
+ | Filename | Quant type | File Size | Description |
24
+ | -------- | ---------- | --------- | ----------- |
25
+ | [blossom-v5-9b-Q8_0.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q8_0.gguf) | Q8_0 | 9.38GB | Extremely high quality, generally unneeded but max available quant. |
26
+ | [blossom-v5-9b-Q6_K.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q6_K.gguf) | Q6_K | 7.24GB | Very high quality, near perfect, *recommended*. |
27
+ | [blossom-v5-9b-Q5_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q5_K_M.gguf) | Q5_K_M | 6.25GB | High quality, very usable. |
28
+ | [blossom-v5-9b-Q5_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q5_K_S.gguf) | Q5_K_S | 6.10GB | High quality, very usable. |
29
+ | [blossom-v5-9b-Q5_0.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q5_0.gguf) | Q5_0 | 6.10GB | High quality, older format, generally not recommended. |
30
+ | [blossom-v5-9b-Q4_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q4_K_M.gguf) | Q4_K_M | 5.32GB | Good quality, similar to 4.25 bpw. |
31
+ | [blossom-v5-9b-Q4_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q4_K_S.gguf) | Q4_K_S | 5.07GB | Slightly lower quality with small space savings. |
32
+ | [blossom-v5-9b-IQ4_NL.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-IQ4_NL.gguf) | IQ4_NL | 5.08GB | Good quality, similar to Q4_K_S, new method of quanting, |
33
+ | [blossom-v5-9b-IQ4_XS.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-IQ4_XS.gguf) | IQ4_XS | 4.82GB | Decent quality, new method with similar performance to Q4. |
34
+ | [blossom-v5-9b-Q4_0.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q4_0.gguf) | Q4_0 | 5.03GB | Decent quality, older format, generally not recommended. |
35
+ | [blossom-v5-9b-IQ3_M.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-IQ3_M.gguf) | IQ3_M | 4.05GB | Medium-low quality, new method with decent performance. |
36
+ | [blossom-v5-9b-IQ3_S.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-IQ3_S.gguf) | IQ3_S | 3.91GB | Lower quality, new method with decent performance, recommended over Q3 quants. |
37
+ | [blossom-v5-9b-Q3_K_L.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q3_K_L.gguf) | Q3_K_L | 4.69GB | Lower quality but usable, good for low RAM availability. |
38
+ | [blossom-v5-9b-Q3_K_M.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q3_K_M.gguf) | Q3_K_M | 4.32GB | Even lower quality. |
39
+ | [blossom-v5-9b-Q3_K_S.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q3_K_S.gguf) | Q3_K_S | 3.89GB | Low quality, not recommended. |
40
+ | [blossom-v5-9b-Q2_K.gguf](https://huggingface.co/bartowski/blossom-v5-9b-GGUF/blob/main/blossom-v5-9b-Q2_K.gguf) | Q2_K | 3.35GB | Extremely low quality, *not* recommended.
41
+
42
+ Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski
blossom-v5-9b-IQ3_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1b1c5a25f3724e1dc733d54f47b10fdd9cb5bc230e980be2677003bcb7a87777
3
+ size 4055461664
blossom-v5-9b-IQ3_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b9e36a8f5031313190006088bde5132afa7a90de88f9354e1f587199f47052c4
3
+ size 3912576800
blossom-v5-9b-IQ4_NL.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae5190014c6ec400f79bf0b66f71284740bc87d30147ece524c45d40e6624e8c
3
+ size 5083393824
blossom-v5-9b-IQ4_XS.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:63ed706869c0b81bb490cf160adb4ca7c8b7a449d07f9e7c0d7f6cfeef0e1218
3
+ size 4827279136
blossom-v5-9b-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27ae54d9c8b03120c639f1a8cb341dec16c19cccdb9403ea5a67015aff5b474f
3
+ size 3354324768
blossom-v5-9b-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:932e6a81ac8104c9b7bc114fdec6eacf38128373a89a4a3166f20c1a155fb846
3
+ size 4690751264
blossom-v5-9b-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:31751a4bb0841ce39cebae5570bc35c5aca93260ac51aa097a0e253ac7d07d92
3
+ size 4324405024
blossom-v5-9b-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:160bb095bb117283fa916793c1b78c5a83af5f82ba1b21c825b6c997aed52f3e
3
+ size 3899207456
blossom-v5-9b-Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0d2733530a04eddc824635329b75531dca3b35b1e5879bf832f8fee6961c0a0e
3
+ size 5036994336
blossom-v5-9b-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7189c08b811204733dd346384145f1efd985d2fc2e324fd6cdf18ce453dd4846
3
+ size 5328957216
blossom-v5-9b-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:52c374ba5ec96257d763c4dfb62a181f467e55483f56c83f1316600de79d4e73
3
+ size 5071859488
blossom-v5-9b-Q5_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e458e309fcb511aa6e4d4f6b73e3d2dcdfd0f059dac1073f6fae93159dd8024f
3
+ size 6107852576
blossom-v5-9b-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9d53a33a116a37eafbafa5ed4503e1d168cb7d090ec82499422ef82e0657655e
3
+ size 6258257696
blossom-v5-9b-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2694c70280b2b5f0cc81b39e217d35923a13dac75527e69bdf808e5dce5aa609
3
+ size 6107852576
blossom-v5-9b-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ae0309afdbe9d38bf373181fb0267dcb7cdc07f3a710fb444050d6117f14e311
3
+ size 7245639456
blossom-v5-9b-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a101dfc70475930c199c2b025e8ef71d99d98ef0091a1d0365628c99345c2e40
3
+ size 9383915296