Update README.md
Browse files
README.md
CHANGED
|
@@ -125,6 +125,7 @@ model-index:
|
|
| 125 |
|
| 126 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 127 |
|
|
|
|
| 128 |
|
| 129 |
## Merge Details
|
| 130 |
|
|
|
|
| 125 |
|
| 126 |
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
|
| 127 |
|
| 128 |
+
I would suggest to play with rope_theta in config.json to set between 40000-100000.
|
| 129 |
|
| 130 |
## Merge Details
|
| 131 |
|