Update README.md
Browse files
README.md
CHANGED
@@ -14,8 +14,9 @@ pipeline_tag: text-generation
|
|
14 |
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T
|
15 |
---
|
16 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65ca8c3c5495933ab066c33c/4iYoUWbwZVIld2K3red-T.png)
|
17 |
-
We, XeTute, introduce AURORA V1.0 -
|
18 |
-
|
|
|
19 |
- Next-Word prediction for mobile devices:
|
20 |
- - This Model can be reliably packaged into a keyboard-app to help make Next-Word suggestions more accurate (for performance, INT4 or less might be smart)
|
21 |
- Conversations:
|
@@ -23,21 +24,21 @@ Its usecases are following:
|
|
23 |
- - AURORA can engage in SFW roleplay with simple character definitions. It wasn't trained on NSFW.
|
24 |
- - AURORA can engage in simple, short Q&A. It was trained on factual data too, which means it performs well for its size.
|
25 |
|
26 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
|
28 |
<a href='https://ko-fi.com/C0C2ZXNON' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>
|
29 |
-
|
30 |
Note:
|
31 |
- All previous beta versions of this series of SLMs were deleted, because almost no downloads were made.
|
32 |
- V1.0 is the last model in this series which will be published, because of too little community activity.
|
33 |
|
34 |
-
Metadata:
|
35 |
-
- Name: AURORA
|
36 |
-
- Version: 1.0
|
37 |
-
- Author: XeTute
|
38 |
-
- Size: 1.1B
|
39 |
-
- Architecture: LaMA, Transformer.
|
40 |
-
|
41 |
Recommended settings:
|
42 |
- Temperature 0.1 - 0,4 is stable.
|
43 |
- Context Length of 2048(base) to 4096(RoPE) will work well for story-telling, role-playing and simple conversations.
|
|
|
14 |
base_model: TinyLlama/TinyLlama-1.1B-intermediate-step-715k-1.5T
|
15 |
---
|
16 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65ca8c3c5495933ab066c33c/4iYoUWbwZVIld2K3red-T.png)
|
17 |
+
We, XeTute, introduce AURORA V1.0 - a humerous, efficient, smart(for its size) and unbiased(because of too low parameters =)) Language Model.
|
18 |
+
|
19 |
+
**Intended usecases:**
|
20 |
- Next-Word prediction for mobile devices:
|
21 |
- - This Model can be reliably packaged into a keyboard-app to help make Next-Word suggestions more accurate (for performance, INT4 or less might be smart)
|
22 |
- Conversations:
|
|
|
24 |
- - AURORA can engage in SFW roleplay with simple character definitions. It wasn't trained on NSFW.
|
25 |
- - AURORA can engage in simple, short Q&A. It was trained on factual data too, which means it performs well for its size.
|
26 |
|
27 |
+
**Training:**
|
28 |
+
- Trained for two months.
|
29 |
+
- Dataset created by XeTute, and translated using different free-lancing services.
|
30 |
+
- Dataset included:
|
31 |
+
- - Mathematic Q&A
|
32 |
+
- - Logic Q&A
|
33 |
+
- - One-Page stories and roleplays with very brief character definitions
|
34 |
+
- ADAM as an optimizer.
|
35 |
+
Alltogether, the model was trained on additional 20B tokens.
|
36 |
|
37 |
<a href='https://ko-fi.com/C0C2ZXNON' target='_blank'><img height='36' style='border:0px;height:36px;' src='https://storage.ko-fi.com/cdn/kofi3.png?v=3' border='0' alt='Buy Me a Coffee at ko-fi.com' /></a>
|
|
|
38 |
Note:
|
39 |
- All previous beta versions of this series of SLMs were deleted, because almost no downloads were made.
|
40 |
- V1.0 is the last model in this series which will be published, because of too little community activity.
|
41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
42 |
Recommended settings:
|
43 |
- Temperature 0.1 - 0,4 is stable.
|
44 |
- Context Length of 2048(base) to 4096(RoPE) will work well for story-telling, role-playing and simple conversations.
|