TheBloke

/

Hermes-LLongMA-2-13B-8K-AWQ

like 1

I'm getting this error while loading in vllm, it's so weird the model is literally named AWQ
ValueError: torch.bfloat16 is not supported for quantization method awq. Supported dtypes: [torch.float16]