neuralmagic
/

Meta-Llama-3.1-70B-Instruct-quantized.w8a16

Text Generation

text-generation-inference

Inference Endpoints

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on Jul 30

Commit

c085fe9

•

1 Parent(s): 17984c5

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -278,7 +278,7 @@ lm_eval \
 lm_eval \
   --model vllm \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=4 \
-  --tasks truthfulqa_mc \
   --num_fewshot 0 \
   --batch_size auto
 ```

 lm_eval \
   --model vllm \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=4 \
+  --tasks truthfulqa \
   --num_fewshot 0 \
   --batch_size auto
 ```