neuralmagic
/

Llama-3.2-1B-Instruct-FP8-dynamic

Text Generation

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on Sep 26

Commit

7549921

•

1 Parent(s): 6407f3b

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -246,7 +246,7 @@ The results were obtained using the following commands:
 ```
 lm_eval \
   --model vllm \
-  --model_args pretrained="neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic",dtype=auto,add_bos_token=True,max_model_len=3850,max_gen_toks=10,tensor_parallel_size=1 \
   --tasks mmlu_llama_3.1_instruct \
   --fewshot_as_multiturn \
   --apply_chat_template \

 ```
 lm_eval \
   --model vllm \
+  --model_args pretrained="neuralmagic/Llama-3.2-1B-Instruct-FP8-dynamic",dtype=auto,max_model_len=3850,max_gen_toks=10,tensor_parallel_size=1 \
   --tasks mmlu_llama_3.1_instruct \
   --fewshot_as_multiturn \
   --apply_chat_template \