Update README.md
Browse files
README.md
CHANGED
@@ -22,19 +22,20 @@ This model is based on Qwen/Qwen1.5-72B and subject to the [tongyi-qianwen](http
|
|
22 |
|
23 |
The base model has 32k context, I finetuned it with 8k.
|
24 |
|
25 |
-
Liberated consists of
|
|
|
|
|
26 |
|
27 |
-
This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models
|
28 |
You are responsible for any content you create using this model. Enjoy responsibly.
|
29 |
|
30 |
## Training
|
31 |
-
It took 3 days to train 3 epochs on 8x H100s using qLoRA and Axolotl
|
32 |
|
33 |
-
Prompt format
|
34 |
This model uses ChatML prompt format.
|
35 |
```
|
36 |
<|im_start|>system
|
37 |
-
You are
|
38 |
<|im_start|>user
|
39 |
{prompt}<|im_end|>
|
40 |
<|im_start|>assistant
|
|
|
22 |
|
23 |
The base model has 32k context, I finetuned it with 8k.
|
24 |
|
25 |
+
Liberated consists of open source datasets, including [SystemChat](https://huggingface.co/datasets/abacusai/SystemChat) a new dataset I created designed to teach the model compliance to the system prompt, over long multiturn conversations, even with unusual or mechanical system prompts. These are tasks that Open Source Models have been lacking in thus far.
|
26 |
+
|
27 |
+
There are no guardrails or censorship added to the dataset. You are advised to implement your own alignment layer before exposing the model as a service. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models
|
28 |
|
|
|
29 |
You are responsible for any content you create using this model. Enjoy responsibly.
|
30 |
|
31 |
## Training
|
32 |
+
It took 3 days to train 3 epochs on 8x H100s using qLoRA, deepspeed zero-2, and Axolotl. learning rate 2e-4.
|
33 |
|
34 |
+
## Prompt format
|
35 |
This model uses ChatML prompt format.
|
36 |
```
|
37 |
<|im_start|>system
|
38 |
+
You are Liberated, a helpful AI assistant.<|im_end|>
|
39 |
<|im_start|>user
|
40 |
{prompt}<|im_end|>
|
41 |
<|im_start|>assistant
|