myeongho-jeong
commited on
Commit
•
8bc753a
1
Parent(s):
498bf08
Update README.md
Browse files
README.md
CHANGED
@@ -4,11 +4,11 @@ base_model: upstage/SOLAR-10.7B-v1.0
|
|
4 |
tags:
|
5 |
- generated_from_trainer
|
6 |
model-index:
|
7 |
-
- name: yanolja/
|
8 |
results: []
|
9 |
---
|
10 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
11 |
-
#
|
12 |
|
13 |
## Join Our Community on Discord!
|
14 |
|
@@ -52,6 +52,8 @@ Our strategy involved a selective freeze of model parameters. Specifically, we k
|
|
52 |
|
53 |
As a result, we froze the internal layers and the first 32,000 `embed_tokens`, directing our training efforts on a rich mix of Korean and multi-lingual corpora. This balanced approach has notably improved the model’s proficiency in Korean, without compromising its original language capabilities.
|
54 |
|
|
|
|
|
55 |
### Usage and Limitations
|
56 |
|
57 |
Keep in mind that this model hasn't been fine-tuned with instruction-based training. While it excels in Korean language tasks, we advise careful consideration and further training for specific applications.
|
@@ -86,11 +88,15 @@ Our model’s training was comprehensive and diverse:
|
|
86 |
|
87 |
This rigorous approach ensured a comprehensive and contextually rich Korean vocabulary for the model.
|
88 |
|
|
|
89 |
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
94 |
-
|
95 |
-
|
96 |
-
|
|
|
|
|
|
|
|
4 |
tags:
|
5 |
- generated_from_trainer
|
6 |
model-index:
|
7 |
+
- name: yanolja/EEVE-Korean-10.8B-v1.0
|
8 |
results: []
|
9 |
---
|
10 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
11 |
+
# EEVE-Korean-10.8B-v1.0
|
12 |
|
13 |
## Join Our Community on Discord!
|
14 |
|
|
|
52 |
|
53 |
As a result, we froze the internal layers and the first 32,000 `embed_tokens`, directing our training efforts on a rich mix of Korean and multi-lingual corpora. This balanced approach has notably improved the model’s proficiency in Korean, without compromising its original language capabilities.
|
54 |
|
55 |
+
For detail, please refer our technical report - [Efficient and Effective Vocabulary Expansion Towards Multilingual Large Language Models](https://arxiv.org).
|
56 |
+
|
57 |
### Usage and Limitations
|
58 |
|
59 |
Keep in mind that this model hasn't been fine-tuned with instruction-based training. While it excels in Korean language tasks, we advise careful consideration and further training for specific applications.
|
|
|
88 |
|
89 |
This rigorous approach ensured a comprehensive and contextually rich Korean vocabulary for the model.
|
90 |
|
91 |
+
## Citation
|
92 |
|
93 |
+
```
|
94 |
+
@misc{cui2023ultrafeedback,
|
95 |
+
title={UltraFeedback: Boosting Language Models with High-quality Feedback},
|
96 |
+
author={Ganqu Cui and Lifan Yuan and Ning Ding and Guanming Yao and Wei Zhu and Yuan Ni and Guotong Xie and Zhiyuan Liu and Maosong Sun},
|
97 |
+
year={2023},
|
98 |
+
eprint={2310.01377},
|
99 |
+
archivePrefix={arXiv},
|
100 |
+
primaryClass={cs.CL}
|
101 |
+
}
|
102 |
+
```
|