Text Generation
Transformers
Safetensors
English
llama
causal-lm
text-generation-inference
4-bit precision
gptq
TheBloke commited on
Commit
1f1a8c1
1 Parent(s): 415a048

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -40,7 +40,7 @@ Load text-generation-webui as you normally do.
40
 
41
  ## GIBBERISH OUTPUT IN `text-generation-webui`?
42
 
43
- Please read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
44
 
45
  If you're using a text-generation-webui one click installer, you MUST use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors`.
46
 
@@ -48,9 +48,9 @@ If you're using a text-generation-webui one click installer, you MUST use `stabl
48
 
49
  Two files are provided. **The 'latest' file will not work unless you use a recent version of GPTQ-for-LLaMa**
50
 
51
- If you do an automatic download with `text-generation-webui` it will pick the 'compat' file which should work for everyone.
52
 
53
- Specifically, the second file uses `--act-order` for maximum quantisation quality and will not work with oobabooga's fork of GPTQ-for-LLaMa. Therefore at this time it will also not work with `text-generation-webui` one-click installers.
54
 
55
  Unless you are able to use the latest GPTQ-for-LLaMa code, please use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors`.
56
 
 
40
 
41
  ## GIBBERISH OUTPUT IN `text-generation-webui`?
42
 
43
+ If you're installing manually, olease read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
44
 
45
  If you're using a text-generation-webui one click installer, you MUST use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors`.
46
 
 
48
 
49
  Two files are provided. **The 'latest' file will not work unless you use a recent version of GPTQ-for-LLaMa**
50
 
51
+ If you do an automatic download with `text-generation-webui` as described above it will pick the 'compat' file which should work for everyone.
52
 
53
+ The 'latest' file uses `--act-order` for maximum quantisation quality and will not work with oobabooga's fork of GPTQ-for-LLaMa. Therefore at this time it will also not work with `text-generation-webui` one-click installers.
54
 
55
  Unless you are able to use the latest GPTQ-for-LLaMa code, please use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors`.
56