Update README.md
Browse files
README.md
CHANGED
|
@@ -18,7 +18,7 @@ This is an instruction fine-tuned llama 30b model, using synthetic data generate
|
|
| 18 |
- Thousands of new generations, using some of the updates re: Flesch hints, etc., to get longer/higher quality writing outputs.
|
| 19 |
- A small "de-alignment" dataset was also added (not published) to remove some of the censorship in the base models.
|
| 20 |
|
| 21 |
-
I used rope scaling to increase context length to 4096, but there wasn't a huge amount of data in that range so YMMV.
|
| 22 |
|
| 23 |
*Why do I try to remove censorship?*
|
| 24 |
|
|
|
|
| 18 |
- Thousands of new generations, using some of the updates re: Flesch hints, etc., to get longer/higher quality writing outputs.
|
| 19 |
- A small "de-alignment" dataset was also added (not published) to remove some of the censorship in the base models.
|
| 20 |
|
| 21 |
+
I used rope scaling to increase context length to 4096, but there wasn't a huge amount of data in that range so YMMV. You may need to rename the directory to `airoboros-33b-2-1` (anything without the `.` really) so the trust_remote_code=True finds the module path properly.
|
| 22 |
|
| 23 |
*Why do I try to remove censorship?*
|
| 24 |
|