Initial GPTQ model commit

README.md

@@ -23,12 +23,14 @@ tags:
 </div>
 <!-- header end -->
 
-# NousResearch's Redmond Puffin 13B GPTQ
+# NousResearch's Redmond Puffin 13B V1.3 GPTQ
 
-These files are GPTQ model files for [NousResearch's Redmond Puffin 13B](https://huggingface.co/NousResearch/Redmond-Puffin-13B).
+These files are GPTQ model files for [NousResearch's Redmond Puffin 13B V1.3](https://huggingface.co/NousResearch/Redmond-Puffin-13B).
 
 Multiple GPTQ parameter permutations are provided; see Provided Files below for details of the options provided, their parameters, and the software used to create them.
 
+Many thanks to William Beauchamp from [Chai](https://chai-research.com/) for providing the hardware used to make and upload these files!
+
 ## Repositories available
 
 * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/Redmond-Puffin-13B-GPTQ)
@@ -50,10 +52,14 @@ Each separate quant is in a different branch.  See below for instructions on fet
 
 | Branch | Bits | Group Size | Act Order (desc_act) | File Size | ExLlama Compatible? | Made With | Description |
 | ------ | ---- | ---------- | -------------------- | --------- | ------------------- | --------- | ----------- |
-| main | 4 | 128 | False | 7.26 GB | True | AutoGPTQ | Most compatible option. Good inference speed in AutoGPTQ and GPTQ-for-LLaMa. Lower inference quality than other options. |
-| gptq-4bit-32g-actorder_True | 4 | 32 | True | 8.00 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 32g gives highest possible inference quality, with maximum VRAM usage. Poor AutoGPTQ CUDA speed. |
-| gptq-4bit-64g-actorder_True | 4 | 64 | True | 7.51 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 64g uses less VRAM than 32g, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
-| gptq-4bit-128g-actorder_True | 4 | 128 | True | 7.26 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. |
+| main | 4 | 128 | False | 7.26 GB | True | AutoGPTQ | Most compatible option. Good inference speed in AutoGPTQ and GPTQ-for-LLaMa. Lower inference quality than other options. | 
+| gptq-4bit-32g-actorder_True | 4 | 32 | True | 8.00 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 32g gives highest possible inference quality, with maximum VRAM usage. Poor AutoGPTQ CUDA speed. | 
+| gptq-4bit-64g-actorder_True | 4 | 64 | True | 7.51 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 64g uses less VRAM than 32g, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. | 
+| gptq-4bit-128g-actorder_True | 4 | 128 | True | 7.26 GB | True | AutoGPTQ | 4-bit, with Act Order and group size. 128g uses even less VRAM, but with slightly lower accuracy. Poor AutoGPTQ CUDA speed. | 
+| gptq-8bit--1g-actorder_True | 8 | None | True | 13.36 GB | False | AutoGPTQ | 8-bit, with Act Order. No group size, to lower VRAM requirements and to improve AutoGPTQ speed. | 
+| gptq-8bit-128g-actorder_False | 8 | 128 | False | 13.65 GB | False | AutoGPTQ | 8-bit, with group size 128g for higher inference quality and without Act Order to improve AutoGPTQ speed. | 
+| gptq-8bit-128g-actorder_True | 8 | 128 | True | 13.65 GB | False | AutoGPTQ | 8-bit, with group size 128g for higher inference quality and with Act Order for even higher accuracy. Poor AutoGPTQ CUDA speed. | 
+| gptq-8bit-64g-actorder_True | 8 | 64 | True | 13.95 GB | False | AutoGPTQ | 8-bit, with group size 64g and Act Order for maximum inference quality. Poor AutoGPTQ CUDA speed. |
 
 ## How to download from branches
 
@@ -103,7 +109,7 @@ use_triton = False
 tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
 
 model = AutoGPTQForCausalLM.from_quantized(model_name_or_path,
-        model_basename=model_basename
+        model_basename=model_basename,
         use_safetensors=True,
         trust_remote_code=False,
         device="cuda:0",
@@ -180,17 +186,19 @@ Donaters will get priority support on any and all AI/LLM/model questions and req
 
 **Special thanks to**: Luke from CarbonQuill, Aemon Algiz.
 
-**Patreon special mentions**: Space Cruiser, Nikolai Manek, Sam, Chris McCloskey, Rishabh Srivastava, Kalila, Spiking Neurons AB, Khalefa Al-Ahmad, WelcomeToTheClub, Chadd, Lone Striker, Viktor Bowallius, Edmond Seymore, Ai Maven, Chris Smitley, Dave, Alexandros Triantafyllidis, Luke @flexchar, Elle, ya boyyy, Talal Aujan, Alex , Jonathan Leane, Deep Realms, Randy H, subjectnull, Preetika Verma, Joseph William Delisle, Michael Levine, chris gileta, K, Oscar Rangel, LangChain4j, Trenton Dambrowitz, Eugene Pentland, Johann-Peter Hartmann, Femi Adebogun, Illia Dulskyi, senxiiz, Daniel P. Andersen, Sean Connelly, Artur Olbinski, RoA, Mano Prime, Derek Yates, Raven Klaugh, David Flickinger, Willem Michiel, Pieter, Willian Hasse, vamX, Luke Pendergrass, webtim, Ghost , Rainer Wilmers, Nathan LeClaire, Will Dee, Cory Kujawski, John Detwiler, Fred von Graf, biorpg, Iucharbius , Imad Khwaja, Pierre Kircher, terasurfer , Asp the Wyvern, John Villwock, theTransient, zynix , Gabriel Tamborski, Fen Risland, Gabriel Puliatti, Matthew Berman, Pyrater, SuperWojo, Stephen Murray, Karl Bernard, Ajan Kanaga, Greatston Gnanesh, Junyu Yang.
+**Patreon special mentions**: Slarti, Chadd, John Detwiler, Pieter, zynix, K, Mano Prime, ReadyPlayerEmma, Ai Maven, Leonard Tan, Edmond Seymore, Joseph William Delisle, Luke @flexchar, Fred von Graf, Viktor Bowallius, Rishabh Srivastava, Nikolai Manek, Matthew Berman, Johann-Peter Hartmann, ya boyyy, Greatston Gnanesh, Femi Adebogun, Talal Aujan, Jonathan Leane, terasurfer, David Flickinger, William Sang, Ajan Kanaga, Vadim, Artur Olbinski, Raven Klaugh, Michael Levine, Oscar Rangel, Randy H, Cory Kujawski, RoA, Dave, Alex, Alexandros Triantafyllidis, Fen Risland, Eugene Pentland, vamX, Elle, Nathan LeClaire, Khalefa Al-Ahmad, Rainer Wilmers, subjectnull, Junyu Yang, Daniel P. Andersen, SuperWojo, LangChain4j, Mandus, Kalila, Illia Dulskyi, Trenton Dambrowitz, Asp the Wyvern, Derek Yates, Jeffrey Morgan, Deep Realms, Imad Khwaja, Pyrater, Preetika Verma, biorpg, Gabriel Tamborski, Stephen Murray, Spiking Neurons AB, Iucharbius, Chris Smitley, Willem Michiel, Luke Pendergrass, Sebastain Graf, senxiiz, Will Dee, Space Cruiser, Karl Bernard, Clay Pascal, Lone Striker, transmissions 11, webtim, WelcomeToTheClub, Sam, theTransient, Pierre Kircher, chris gileta, John Villwock, Sean Connelly, Willian Hasse
+
 
 Thank you to all my generous patrons and donaters!
 
 <!-- footer end -->
 
-# Original model card: NousResearch's Redmond Puffin 13B
+# Original model card: NousResearch's Redmond Puffin 13B V1.3
+
 
 ![puffin](https://i.imgur.com/R2xTHMb.png)
 
-## **Redmond-Puffin-13b (Currently available as a Preview edition)**
+## **Redmond-Puffin-13b-V1.3**
 
 **The first commercially available language model released by Nous Research!**
 
@@ -204,25 +212,39 @@ Notable mentions for assisting in some of the training issues goes to: Caseus an
 
 ## Model Training
 
-Redmond-Puffin-13B is a new model trained for multiple epochs on a dataset of 3,000 carefully curated GPT-4 examples, most of which are long context conversations between a real human and GPT-4. 
+Redmond-Puffin-13B-V1.3 is a new model trained for multiple epochs on a dataset of 3,000 carefully curated GPT-4 examples, most of which are long context conversations between a real human and GPT-4. 
 
-Additional data came from carefully curated subsections of datasets such as CamelAI's Physics, Chemistry, Biology and Math.
+Additional data came from carefully curated sub sections of datasets such as CamelAI's Physics, Chemistry, Biology and Math.
 
 ## Prompt Format
 
-The model follows the Vicuna ShareGPT prompt format:
+The reccomended model usage is:
 
 ```
 ### human:
 
-### gpt:
+### response:
+
+```
+Optional reccomended pre-prompt / system prompt:
+
 ```
+### human: Interact in conversation to the best of your ability, please be concise, logical, intelligent and coherent.
+
+### response: Sure! sounds good.
+```
+
+## Improvements over previous version:
+
+The original Puffin model was loved by many, however it was quickly discovered to have dataset errors in a significant amount of the conversations.
+Puffin-V1.3 dataset solves this issue and the resulting fixed model has now fully finished training!
+
 
 ## Notable Features:
 
  - The first Llama-2 based fine-tuned model released by Nous Research.
 
- - Ability to recall information from upto late 2022 without internet. (ChatGPT cut off date is in 2021)
+ - Ability to recall information upto 2023 without internet (ChatGPT cut off date is in 2021)
 
  - Pretrained on 2 trillion tokens of text. (This is double the amount of most Open LLM's)
 
@@ -240,9 +262,13 @@ We plan to have these solved in an updated Puffin model in the very near future,
 
 This is a relatively early build amongst the grand plans for the future of Puffin! 
 
-Current limitations: Some token mismatch problems and formatting issues have been idenitifed, these may very possibly effect the current output quality, we plan to have these solved in an updated Puffin model in the near future.
+Current limitations: Some token mismatch problems have been identified, these may effect the current output quality, we plan to have this solved in Puffin V2 along with other improvements.
+
+## How you can help!
+
+In the near future we plan on leveraging the help of domain specific expert volunteers to eliminate any mathematically/verifiably incorrect answers from our training curations. 
 
-In the near future we plan on releasing an improved version of the model with the help of domain specific expert volunteers, which will help eliminate any wrong data from this curation and improve the further ones.
+If you have at-least a bachelors in mathematics, physics, biology or chemistry and would like to volunteer even just 30 minutes of your expertise time, please contact ldj on discord!
 
 ## Benchmarks coming soon