mradermacher
/

Qwen2.5-Coder-1.5B-Instruct-i1-GGUF

Inference Endpoints

Model card Files Files and versions Community

mradermacher commited on 13 days ago

Commit

385165c

•

1 Parent(s): 66a3cb4

auto-patch README.md

Files changed (1) hide show

README.md +1 -0

README.md CHANGED Viewed

@@ -38,6 +38,7 @@ more details, including on how to concatenate multi-part files.
 |:-----|:-----|--------:|:------|
 | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-1.5B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 0.8 | IQ3_XXS probably better |
 | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-1.5B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 0.9 |  |
 Here is a handy graph by ikawrakow comparing some lower-quality quant
 types (lower is better):

 |:-----|:-----|--------:|:------|
 | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-1.5B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct.i1-Q2_K.gguf) | i1-Q2_K | 0.8 | IQ3_XXS probably better |
 | [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-1.5B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct.i1-IQ3_M.gguf) | i1-IQ3_M | 0.9 |  |
+| [GGUF](https://huggingface.co/mradermacher/Qwen2.5-Coder-1.5B-Instruct-i1-GGUF/resolve/main/Qwen2.5-Coder-1.5B-Instruct.i1-Q4_K_S.gguf) | i1-Q4_K_S | 1.0 | optimal size/speed/quality |
 Here is a handy graph by ikawrakow comparing some lower-quality quant
 types (lower is better):