add 600B checkpoints

README.md

@@ -14,7 +14,7 @@ datasets:
 
 _**NOTE:** This is a **research checkpoint** of a model for which **training has not been completed.** It is being provided in its current state for research and testing purposes. **Care should be taken when using the outputs of the model.** Once pretraining has completed we intend to release additional instruction-tuned and chat-tuned varieties._
 
-Poro is a 34B parameter decoder-only transformer pretrained on Finnish, English and code.  It is being trained on 1 trillion tokens (500 billion as of this release). Poro is a fully open source model and is made available under the Apache 2.0 License.
+Poro is a 34B parameter decoder-only transformer pretrained on Finnish, English and code.  It is being trained on 1 trillion tokens (600 billion as of this release). Poro is a fully open source model and is made available under the Apache 2.0 License.
 
 Poro was created in a collaboration between [SiloGen](https://www.silo.ai/silogen) from [Silo AI](https://www.silo.ai/), the [TurkuNLP group](https://turkunlp.org/) of the University of Turku, and [High Performance Language Technologies](https://hplt-project.org/) (HPLT). Training was conducted on the [LUMI supercomputer](https://www.lumi-supercomputer.eu/), using compute resources generously provided by [CSC](https://csc.fi/) - IT Center for Science, Finland.
 
@@ -47,6 +47,7 @@ Checkpoints are available as branches in the repository.  Checkpoints will be re
 * [300B](https://huggingface.co/LumiOpen/Poro-34B/tree/300B)
 * [400B](https://huggingface.co/LumiOpen/Poro-34B/tree/400B)
 * [500B](https://huggingface.co/LumiOpen/Poro-34B/tree/500B)
+* [600B](https://huggingface.co/LumiOpen/Poro-34B/tree/600B)
 
 The transformers library allows you to load a checkpoint from a branch as follows:
 

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "/scratch/project_462000319/general-tools/checkpoints/33B_torch_step119232_bfloat16",
+  "_name_or_path": "/scratch/project_462000319/general-tools/checkpoints/33B_torch_step143712_bfloat16",
   "apply_residual_connection_post_layernorm": false,
   "architectures": [
     "BloomForCausalLM"

model-00001-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:48c0ca766875ab7c6c34097e366e6dc5c7b3c3e2048586229bbf69fff98e66b0
+oid sha256:122e8a9baec708629e3476dd789bd16a669ceb461c4811af61f8238179892a2c
 size 4712820784

model-00002-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e0c38f242015ea4fea3122bb0799fe31687d3b28e389e3b8c0bf6e12230681e4
+oid sha256:2689231237b8bbc0b7d1217690de3d90f859966c6fbb881b0827ee38a79ffdb7
 size 4933252680

model-00003-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ad85214905f08989fe3188395dcbc556cdd30651ef662cbb2c3fac63c5425e84
+oid sha256:c1db3f74bdf45dd015aff31edc083d1c7b740a2d18d51524ced786f5f81dfa19
 size 4933252648

model-00004-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e0012b88811f0c150ad2db527e8738140c77c6370cb337064ec745a3d78a7d37
+oid sha256:b006e28ebe97c8416e90d552838ea6e0bbfae1cdaa9de6ab3287509e0d981fbb
 size 4933252728

model-00005-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0cee54d10dd545368cf2ded5397b12c9bed3a1e0cf92bb2c0125848f87cc6d9c
+oid sha256:0486a7f51f2e5a83383c4f436082e6b4e0320642ddda25aaa791b5030e3e4e70
 size 4933252728

model-00006-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a04a2e58e6e625b72a16f10c349bb80fdaac6852c4e1dd51e28818d21251cf95
+oid sha256:1aec6faf7cbeb1a9755844e1e6d888c56e51f9392d7f49110e68c44a4c708383
 size 4933252728

model-00007-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9729d6c473dca448f84e5660ce4d4adb5edcd1afd7ea8f0733ad6a2f69c3863e
+oid sha256:a5f45a9259e64aad76dbe7e4c586bf6575b54faf0f900fff8a51f17439aed691
 size 4933252728

model-00008-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4826ae9a6a655c10e7fbf2fb129665d0961a47cb8c77023859278976019ec46
+oid sha256:caa0d3ced1171735eec853bfd05212c5778d7157a89fbfef4f43c1bb353c18d4
 size 4933252728

model-00009-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:bbae3d11df4f09e8fe3e1ff292187493023ae5a783b620a65cdf5a0a66500b73
+oid sha256:6bce6dd88d67171b89babd472dda1a3e5bc3af8ef0b0714cb96a11090bb28df7
 size 4933252728

model-00010-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:960c2142d7b5c5a05ee10ba44ca5984b89f682583c9a84ee469120872acd837c
+oid sha256:cc28d2be3e6ae211f747c1b9838e357f273b8944b7f3c6e08472546607383554
 size 4933252728

model-00011-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:941e5d4a2e133735be5a21a271f276f8042058cde9d1f2c1ba4029ff3988d965
+oid sha256:0fb88920dd5b872dadc35b0b0dfceb7196bc00c1e5f29d463a932de912ab9a58
 size 4933252728

model-00012-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:405edc4e9c335b535516593c12b6936694d492e286a3edc4007255cbb4797173
+oid sha256:4bf10456a80691d04b4d9734d7adf3c62437b15720c61023d76d013b77f0f6cc
 size 4933252728

model-00013-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4096fcc32fecd609b36e783ecc0c0c01f726e6313d2cd8acde50ef6d93710c06
+oid sha256:792d81f54fc1398d38e7cb415b3ee96c08944b12dc7cec79ea9e8b49546581e6
 size 4933252728

model-00014-of-00014.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91e346a7af6ea5bc28558babc7d982f47c0bd8a2f750fa2f6cc180f41dde55ec
+oid sha256:76bdca2059826a6775c40650558217028f37ad640eef2039a952eac7ba613baa
 size 4522124144