jonabur
commited on
Commit
•
564a58e
1
Parent(s):
8502bcf
add HPLT acknowledgment
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ _**NOTE:** This is a **research checkpoint** of a model for which **training has
|
|
16 |
|
17 |
Poro is a 34B parameter decoder-only transformer pretrained on Finnish, English and code. It is being trained on 1 trillion tokens (300 billion as of this release). Poro is a fully open source model and is made available under the Apache 2.0 License.
|
18 |
|
19 |
-
Poro was created in a collaboration between SiloGen from Silo AI
|
20 |
|
21 |
This project is part of an ongoing effort to create open source large language models for non-English and especially low resource languages like Finnish. Through the combination of English and Finnish training data we get a model that outperforms previous Finnish only models, while also being fluent in English and code, and capable of basic translation between English and Finnish.
|
22 |
|
|
|
16 |
|
17 |
Poro is a 34B parameter decoder-only transformer pretrained on Finnish, English and code. It is being trained on 1 trillion tokens (300 billion as of this release). Poro is a fully open source model and is made available under the Apache 2.0 License.
|
18 |
|
19 |
+
Poro was created in a collaboration between SiloGen from Silo AI, the TurkuNLP group of the University of Turku, and High Performance Language Technologies (HPLT). Training was conducted on the LUMI supercomputer, using compute resources generously provided by CSC - IT Center for Science, Finland.
|
20 |
|
21 |
This project is part of an ongoing effort to create open source large language models for non-English and especially low resource languages like Finnish. Through the combination of English and Finnish training data we get a model that outperforms previous Finnish only models, while also being fluent in English and code, and capable of basic translation between English and Finnish.
|
22 |
|