TheBloke commited on
Commit
609f4a9
1 Parent(s): b866c46

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -45,9 +45,9 @@ Details of the files provided:
45
 
46
  File `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
47
 
48
- The other two model files were created with the latest GPTQ code, and require that the latest GPTQ-for-LLaMa is used inside the UI.
49
 
50
- Here are the commands I used to clone the Triton branch of GPTQ-for-LLaMa, clone text-generation-webui, and install GPTQ into the UI:
51
  ```
52
  git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa
53
  git clone https://github.com/oobabooga/text-generation-webui
@@ -71,7 +71,7 @@ python setup_cuda.py install
71
  ```
72
  Then link that into `text-generation-webui/repositories` as described above.
73
 
74
- Or just use `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` as mentioned above.
75
 
76
  # Vicuna Model Card
77
 
 
45
 
46
  File `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
47
 
48
+ The `safetensors` model file was created with the latest GPTQ code, and uses `--act-order` to give the maximum possible quantisation quality, but this means it requires that the latest GPTQ-for-LLaMa is used inside the UI.
49
 
50
+ If you want to use the `safetensors` file and need to update GPTQ-for-LLaMa, here are the commands I used to clone the Triton branch of GPTQ-for-LLaMa, clone text-generation-webui, and install GPTQ into the UI:
51
  ```
52
  git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa
53
  git clone https://github.com/oobabooga/text-generation-webui
 
71
  ```
72
  Then link that into `text-generation-webui/repositories` as described above.
73
 
74
+ Or just use `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` as mentioned above, which should work without any upgrades to text-generation-webui.
75
 
76
  # Vicuna Model Card
77