Initial GPTQ model commit

README.md

@@ -1,4 +1,14 @@
 ---
+extra_gated_button_content: Submit
+extra_gated_description: This is a form to enable access to Llama 2 on Hugging Face
+  after you have been granted access from Meta. Please visit the [Meta website](https://ai.meta.com/resources/models-and-libraries/llama-downloads)
+  and accept our license terms and acceptable use policy before submitting this form.
+  Requests will be processed in 1-2 days.
+extra_gated_fields:
+  ? I agree to share my name, email address and username with Meta and confirm that
+    I have already been granted download access on the Meta website
+  : checkbox
+extra_gated_heading: Access Llama 2 on Hugging Face
 inference: false
 language:
 - en
@@ -37,7 +47,7 @@ Multiple GPTQ parameter permutations are provided; see Provided Files below for
 
 * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/Llama-2-13B-GPTQ)
 * [2, 3, 4, 5, 6 and 8-bit GGML models for CPU+GPU inference](https://huggingface.co/TheBloke/Llama-2-13B-GGML)
-* [Original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/meta-llama/Llama-2-13b-hf)
+* [Unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/meta-llama/Llama-2-13b-hf)
 
 ## Prompt template: None
 
@@ -56,7 +66,7 @@ Each separate quant is in a different branch.  See below for instructions on fet
 | main | 4 | 128 | False | 7.26 GB | True | AutoGPTQ | Most compatible option. Good inference speed in AutoGPTQ and GPTQ-for-LLaMa. Lower inference quality than other options. |
 | gptq-4bit-32g-actorder_True | 4 | 32 | True | 8.00 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 32g gives highest possible inference quality, with maximum VRAM usage. Poor AutoGPTQ CUDA speed. |
 | gptq-4bit-64g-actorder_True | 4 | 64 | True | 7.51 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 64g uses less VRAM than 32g, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
-| gptq-4bit-128g-actorder_True | 4 | 128 | True | TBC| True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
+| gptq-4bit-128g-actorder_True | 4 | 128 | True | 7.26 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
 
 ## How to download from branches