ehartford commited on
Commit
f0e0a9e
1 Parent(s): fa231e8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -22,19 +22,20 @@ This model is based on Qwen/Qwen1.5-72B and subject to the [tongyi-qianwen](http
22
 
23
  The base model has 32k context, I finetuned it with 8k.
24
 
25
- Liberated consists of fi
 
 
26
 
27
- This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models
28
  You are responsible for any content you create using this model. Enjoy responsibly.
29
 
30
  ## Training
31
- It took 3 days to train 3 epochs on 8x H100s using qLoRA and Axolotl
32
 
33
- Prompt format:
34
  This model uses ChatML prompt format.
35
  ```
36
  <|im_start|>system
37
- You are Dolphin, a helpful AI assistant.<|im_end|>
38
  <|im_start|>user
39
  {prompt}<|im_end|>
40
  <|im_start|>assistant
 
22
 
23
  The base model has 32k context, I finetuned it with 8k.
24
 
25
+ Liberated consists of open source datasets, including [SystemChat](https://huggingface.co/datasets/abacusai/SystemChat) a new dataset I created designed to teach the model compliance to the system prompt, over long multiturn conversations, even with unusual or mechanical system prompts. These are tasks that Open Source Models have been lacking in thus far.
26
+
27
+ There are no guardrails or censorship added to the dataset. You are advised to implement your own alignment layer before exposing the model as a service. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models
28
 
 
29
  You are responsible for any content you create using this model. Enjoy responsibly.
30
 
31
  ## Training
32
+ It took 3 days to train 3 epochs on 8x H100s using qLoRA, deepspeed zero-2, and Axolotl. learning rate 2e-4.
33
 
34
+ ## Prompt format
35
  This model uses ChatML prompt format.
36
  ```
37
  <|im_start|>system
38
+ You are Liberated, a helpful AI assistant.<|im_end|>
39
  <|im_start|>user
40
  {prompt}<|im_end|>
41
  <|im_start|>assistant