Update README.md

README.md

@@ -1,4 +1,5 @@
 ---
+inference: false
 license: mit
 base_model: microsoft/phi-2
 tags:
@@ -7,10 +8,23 @@ tags:
 model-index:
 - name: Phasmid-2_v2
   results: []
+datasets:
+- PygmalionAI/PIPPA
+- HuggingFaceH4/no_robots
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
+
+```
+   _ (`-.  ('-. .-.   ('-.      .-')   _   .-')            _ .-') _  
+  ( (OO  )( OO )  /  ( OO ).-. ( OO ).( '.( OO )_         ( (  OO) )  
+ _.`     \,--. ,--.  / . --. /(_)---\_),--.   ,--.) ,-.-') \     .'_  
+(__...--''|  | |  |  | \-.  \ /    _ | |   `.'   |  |  |OO),`'--..._)
+ |  /  | ||   .|  |.-'-'  |  |\  :` `. |         |  |  |  \|  |  \  '  
+ |  |_.' ||       | \| |_.'  | '..`''.)|  |'.'|  |  |  |(_/|  |   ' |  
+ |  .___.'|  .-.  |  |  .-.  |.-._)   \|  |   |  | ,|  |_.'|  |   / :  
+ |  |     |  | |  |  |  | |  |\       /|  |   |  |(_|  |   |  '--'  / 
+ `--'     `--' `--'  `--' `--' `-----' `--'   `--'  `--'   `-------'  
+```
 
 [<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
 <details><summary>See axolotl config</summary>
@@ -99,23 +113,49 @@ special_tokens:
 
 </details><br>
 
+
 # Phasmid-2_v2
 
-This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on the None dataset.
+This model is a fine-tuned version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2) on a mix of no_robots and the PIPPA dataset.
 It achieves the following results on the evaluation set:
 - Loss: 2.2924
 
 ## Model description
-
-More information needed
-
+Phasmid-2 has been trained on intructional data and thus can perform far better at instruction following than phi-2. However I have not extensively tested the model.
 ## Intended uses & limitations
-
-More information needed
-
-## Training and evaluation data
-
-More information needed
+This model is little more than a side project and I shall treat it as such. 
+Phasmid-2 (due to it's size), can still suffer from problematic hallucinations and poor information. No effort was made to reduce potentially toxic responses, as such you should train this model further if you require it to do so.
+## Inference
+Phi doesn't like device_map = auto, therefore you should specify as like the following:
+
+1. FP16 / Flash-Attention / CUDA:
+   ```python
+   model = AutoModelForCausalLM.from_pretrained("SE6446/Phasmid-2", torch_dtype="auto", flash_attn=True, flash_rotary=True, fused_dense=True, device_map="cuda", trust_remote_code=True)
+   ```
+2. FP16 / CUDA:
+   ```python
+   model = AutoModelForCausalLM.from_pretrained("SE6446/Phasmid-2", torch_dtype="auto", device_map="cuda", trust_remote_code=True)
+   ```
+3. FP32 / CUDA:
+   ```python
+   model = AutoModelForCausalLM.from_pretrained("SE6446/Phasmid-2", torch_dtype=torch.float32, device_map="cuda", trust_remote_code=True)
+   ```
+4. FP32 / CPU:
+   ```python
+   model = AutoModelForCausalLM.from_pretrained("SE6446/Phasmid-2", torch_dtype=torch.float32, device_map="cpu", trust_remote_code=True)
+   ```
+
+And then use the following snippet
+```python
+tokenizer = AutoTokenizer.from_pretrained("SE6446/Phasmid-1_5-V0_1", trust_remote_code=True, torch_dtype="auto")
+inputs = tokenizer('''SYSTEM: You are a helpful assistant. Please answer truthfully and write out your thinking step by step to be sure you get the right answer. If you make a mistake or encounter an error in your thinking, say so out loud and attempt to correct it. If you don't know or aren't sure about something, say so clearly. You will act as a professional logician, mathematician, and physicist. You will also act as the most appropriate type of expert to answer any particular question or solve the relevant problem; state which expert type your are, if so. Also think of any particular named expert that would be ideal to answer the relevant question or solve the relevant problem; name and act as them, if appropriate. (add your custom prompt like a character description in here)\n
+                      USER: {{userinput}}\n
+                      ASSISTANT: {{character name if applicable}}:''', return_tensors="pt", return_attention_mask=False)
+outputs = model.generate(**inputs, max_length=200)
+text = tokenizer.batch_decode(outputs)[0]
+print(text)
+```
+it should generate after "ASSISTANT:".
 
 ## Training procedure
 
@@ -158,4 +198,4 @@ The following hyperparameters were used during training:
 - Transformers 4.37.0.dev0
 - Pytorch 2.0.1+cu118
 - Datasets 2.16.1
-- Tokenizers 0.15.0
+- Tokenizers 0.15.0