TheBloke

/

Llama-2-70B-Chat-GPTQ

like 259

I'm getting error:

Traceback (most recent call last):
  File "app.py", line 11, in <module>
    model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/auto.py", line 94, in from_quantized
    return quant_func(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/_base.py", line 749, in from_quantized
    make_quant(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/_utils.py", line 92, in make_quant
    make_quant(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/_utils.py", line 92, in make_quant
    make_quant(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/_utils.py", line 92, in make_quant
    make_quant(
  [Previous line repeated 1 more time]
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/modeling/_utils.py", line 84, in make_quant
    new_layer = QuantLinear(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/auto_gptq/nn_modules/qlinear/qlinear_cuda_old.py", line 83, in __init__
    self.autogptq_cuda = autogptq_cuda_256
NameError: name 'autogptq_cuda_256' is not defined```

How to fix it?