Update README.md
Browse files
README.md
CHANGED
@@ -38,7 +38,7 @@ About XT_AURORA:
|
|
38 |
XT_AURORA is a series of SLMs[Slender Language Models], which all aim to provide a friendly, human-like conversation.
|
39 |
The serie is limited by its size[about 1.1B Params], but we still try to get the best possible output.
|
40 |
The context-length is very stable till 2048 tokens, after that limit, it will perform only slightly better than V0.5.
|
41 |
-
|
42 |
|
43 |
About this version[V1.0]:
|
44 |
* High quality output[sometimes outperforms 3B models in HumanEval], as long as the context size is under 2049 Tokens.
|
|
|
38 |
XT_AURORA is a series of SLMs[Slender Language Models], which all aim to provide a friendly, human-like conversation.
|
39 |
The serie is limited by its size[about 1.1B Params], but we still try to get the best possible output.
|
40 |
The context-length is very stable till 2048 tokens, after that limit, it will perform only slightly better than V0.5.
|
41 |
+
It can be upscaled using rope, with the cost being slightly less logic.
|
42 |
|
43 |
About this version[V1.0]:
|
44 |
* High quality output[sometimes outperforms 3B models in HumanEval], as long as the context size is under 2049 Tokens.
|