Taishi-N324
commited on
Commit
•
d736d04
1
Parent(s):
9629a9f
Upload README.md
Browse files
README.md
CHANGED
@@ -40,7 +40,7 @@ We are excited to share the release schedule for our latest models:
|
|
40 |
![logo](./logo.png)
|
41 |
|
42 |
This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
|
43 |
-
Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://
|
44 |
|
45 |
## Model Details
|
46 |
|
@@ -224,7 +224,7 @@ The following datasets were used for continual pre-training.
|
|
224 |
|
225 |
- [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
|
226 |
- [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
|
227 |
-
- Swallow Corpus
|
228 |
- [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
|
229 |
|
230 |
|
@@ -265,3 +265,15 @@ Here are the team members:
|
|
265 |
- [Rio Yokota](https://twitter.com/rioyokota)
|
266 |
- [Kazuki Fujii](https://twitter.com/okoge_kaz)
|
267 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
![logo](./logo.png)
|
41 |
|
42 |
This repository provides large language models developed by [TokyoTech-LLM](https://tokyotech-llm.github.io/).
|
43 |
+
Read our [blog post](https://zenn.dev/tokyotech_lm/articles/d6cb3a8fdfc907) or our [paper](https://arxiv.org/abs/2404.17790)
|
44 |
|
45 |
## Model Details
|
46 |
|
|
|
224 |
|
225 |
- [Japanese Wikipedia](https://dumps.wikimedia.org/other/cirrussearch)
|
226 |
- [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb)
|
227 |
+
- [Swallow Corpus](https://arxiv.org/abs/2404.17733)
|
228 |
- [The Pile](https://huggingface.co/datasets/EleutherAI/pile)
|
229 |
|
230 |
|
|
|
265 |
- [Rio Yokota](https://twitter.com/rioyokota)
|
266 |
- [Kazuki Fujii](https://twitter.com/okoge_kaz)
|
267 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
268 |
+
|
269 |
+
## How to cite
|
270 |
+
```
|
271 |
+
@misc{fujii2024continual,
|
272 |
+
title={Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities},
|
273 |
+
author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae Mizuki and Rio Yokota and Naoaki Okazaki},
|
274 |
+
year={2024},
|
275 |
+
eprint={2404.17790},
|
276 |
+
archivePrefix={arXiv},
|
277 |
+
primaryClass={cs.CL}
|
278 |
+
}
|
279 |
+
```
|