Hugging Face's logo

EricB
/
Llama-3.2-11B-Vision-Instruct-UQFF

mllama
uqff
mistral.rs
Model card Files Community
EricB HF staff commited on
Commit
c37b34f
•
1 Parent(s): 41db3dd

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -7
README.md CHANGED
@@ -22,10 +22,10 @@ Run with [mistral.rs](https://github.com/EricLBuehler/mistral.rs). Documentation
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
- |FP8|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-f8e4m3.uqff`|
26
- |HQQ4|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-hqq4.uqff`|
27
- |HQQ8|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-hqq8.uqff`|
28
- |Q3K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-q3k.uqff`|
29
- |Q4K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-q4k.uqff`|
30
- |Q5K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-q5k.uqff`|
31
- |Q8_0|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llam3.2-vision-instruct-q8_0.uqff`|
 
22
 
23
  |Quantization type(s)|Example|
24
  |--|--|
25
+ |FP8|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-f8e4m3.uqff`|
26
+ |HQQ4|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-hqq4.uqff`|
27
+ |HQQ8|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-hqq8.uqff`|
28
+ |Q3K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-q3k.uqff`|
29
+ |Q4K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-q4k.uqff`|
30
+ |Q5K|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-q5k.uqff`|
31
+ |Q8_0|`./mistralrs-server -i vision-plain -m EricB/Llama-3.2-11B-Vision-Instruct-UQFF -a vllama --from-uqff llama3.2-vision-instruct-q8_0.uqff`|