Update README.md
Browse files
README.md
CHANGED
|
@@ -7,6 +7,9 @@ tags:
|
|
| 7 |
|
| 8 |
---
|
| 9 |
# Merge_XL_model_Stock
|
|
|
|
|
|
|
|
|
|
| 10 |
This model switches to the Smaug instruct 32K for the base bodel.
|
| 11 |
Expanded with Giraffe and Gradient to keep a robuust long context window.
|
| 12 |
Higgs and cat for most of the story and RP aspects.
|
|
@@ -43,4 +46,19 @@ models:
|
|
| 43 |
merge_method: model_stock
|
| 44 |
base_model: \Smaug-Llama-3-70B-Instruct-32K
|
| 45 |
dtype: bfloat16
|
| 46 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
|
| 8 |
---
|
| 9 |
# Merge_XL_model_Stock
|
| 10 |
+
Ofcourse the model is still fully focussed on long context Roleplay and Story.
|
| 11 |
+
By far the best itteration.
|
| 12 |
+
|
| 13 |
This model switches to the Smaug instruct 32K for the base bodel.
|
| 14 |
Expanded with Giraffe and Gradient to keep a robuust long context window.
|
| 15 |
Higgs and cat for most of the story and RP aspects.
|
|
|
|
| 46 |
merge_method: model_stock
|
| 47 |
base_model: \Smaug-Llama-3-70B-Instruct-32K
|
| 48 |
dtype: bfloat16
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Any suggestions are very welcome
|
| 52 |
+
My personal sampling settings are:
|
| 53 |
+
"temp": 1,
|
| 54 |
+
"temperature_last": true,
|
| 55 |
+
"top_p": 1,
|
| 56 |
+
"top_k": 0,
|
| 57 |
+
"top_a": 0,
|
| 58 |
+
"tfs": 1,
|
| 59 |
+
"typical_p": 1,
|
| 60 |
+
"min_p": 0.05,
|
| 61 |
+
"rep_pen": 1.05,
|
| 62 |
+
"rep_pen_range": 4096,
|
| 63 |
+
"rep_pen_decay": 0,
|
| 64 |
+
"rep_pen_slope": 1,
|