Update README.md
Browse files
README.md
CHANGED
@@ -45,9 +45,9 @@ Details of the files provided:
|
|
45 |
|
46 |
File `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
|
47 |
|
48 |
-
The
|
49 |
|
50 |
-
|
51 |
```
|
52 |
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa
|
53 |
git clone https://github.com/oobabooga/text-generation-webui
|
@@ -71,7 +71,7 @@ python setup_cuda.py install
|
|
71 |
```
|
72 |
Then link that into `text-generation-webui/repositories` as described above.
|
73 |
|
74 |
-
Or just use `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` as mentioned above.
|
75 |
|
76 |
# Vicuna Model Card
|
77 |
|
|
|
45 |
|
46 |
File `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
|
47 |
|
48 |
+
The `safetensors` model file was created with the latest GPTQ code, and uses `--act-order` to give the maximum possible quantisation quality, but this means it requires that the latest GPTQ-for-LLaMa is used inside the UI.
|
49 |
|
50 |
+
If you want to use the `safetensors` file and need to update GPTQ-for-LLaMa, here are the commands I used to clone the Triton branch of GPTQ-for-LLaMa, clone text-generation-webui, and install GPTQ into the UI:
|
51 |
```
|
52 |
git clone https://github.com/qwopqwop200/GPTQ-for-LLaMa
|
53 |
git clone https://github.com/oobabooga/text-generation-webui
|
|
|
71 |
```
|
72 |
Then link that into `text-generation-webui/repositories` as described above.
|
73 |
|
74 |
+
Or just use `vicuna-13B-1.1-GPTQ-4bit-128g.no-act-order.pt` as mentioned above, which should work without any upgrades to text-generation-webui.
|
75 |
|
76 |
# Vicuna Model Card
|
77 |
|