sokada commited on
Commit
6344b73
1 Parent(s): 0c0c559

update README

Browse files
Files changed (1) hide show
  1. README.md +18 -9
README.md CHANGED
@@ -1,23 +1,31 @@
 
 
 
 
 
 
 
 
 
1
  # PLaMo-13B
2
 
3
  ## Model Description
4
- PLaMo-13B is a Llama 13B model pre-trained on English and Japanese open datasets, developed by Preferred Networks, Inc.
5
  PLaMo-13B is released under Apache v2.0 license.
6
 
7
- PLaMo-13BはPreferred Networks, Incが英語と日本語の公開データセットで訓練したLlama-13Bモデルです。
8
- PLaMo-13BはApache v2.0ライセンスでリリースされています。
9
 
10
  ## Usage
11
 
12
  ### Use a pipeline as a high-level helper
13
- ```
14
  import transformers
15
  pipeline = transformers.pipeline("text-generation", model="pfnet/plamo-13b", trust_remote_code=True)
16
  print(pipeline("The future of artificial intelligence technology is ", max_new_tokens=32))
17
  ```
18
 
19
  ### Load model directly
20
- ```
21
  from transformers import AutoTokenizer, AutoModelForCausalLM
22
  tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
23
  model = AutoModelForCausalLM.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
@@ -31,14 +39,15 @@ generated_tokens = model.generate(
31
  top_p=0.95,
32
  temperature=1.0,
33
  )[0]
34
- print(tokenizer.decode(generated_tokens))
35
-
36
  ```
37
 
38
  ## Model Details
39
 
40
  - Model size: 13B
41
  - Trained tokens: 1.5T tokens (English: 1.32T tokens, Japanese: 0.18T tokens)
 
42
  - Developed by: Preferred Networkfs, Inc
43
  - Model type: Causal decoder-only
44
  - Language(s): English, Japanese
@@ -68,7 +77,7 @@ PLaMo-13B uses sentencepiece tokenizer which is trained on a subset of the datas
68
  PLaMo-13B is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-13B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-13B, developers should perform safety testing and tuning tailored to their specific applications of the model.
69
 
70
  ## How to cite
71
- ```
72
  @online{PLaMo2023Introducing,
73
  author = {Preferred Networks, Inc},
74
  title = {PLaMo-13B},
@@ -79,7 +88,7 @@ PLaMo-13B is a new technology that carries risks with use. Testing conducted to
79
  ```
80
 
81
  ## Citations
82
- ```
83
  @article{touvron2023llama,
84
  title={LLaMA: Open and Efficient Foundation Language Models},
85
  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},
 
1
+ ---
2
+ language:
3
+ - en
4
+ - ja
5
+ license: apache-2.0
6
+ library_name: transformers
7
+ pipeline_tag: text-generation
8
+ ---
9
+
10
  # PLaMo-13B
11
 
12
  ## Model Description
13
+ PLaMo-13B is a LLaMA-based 13B model pre-trained on English and Japanese open datasets, developed by Preferred Networks, Inc.
14
  PLaMo-13B is released under Apache v2.0 license.
15
 
16
+ [PLaMo-13B Release blog (Japanese)](https://tech.preferred.jp/ja/blog/llm-plamo/)
 
17
 
18
  ## Usage
19
 
20
  ### Use a pipeline as a high-level helper
21
+ ```python
22
  import transformers
23
  pipeline = transformers.pipeline("text-generation", model="pfnet/plamo-13b", trust_remote_code=True)
24
  print(pipeline("The future of artificial intelligence technology is ", max_new_tokens=32))
25
  ```
26
 
27
  ### Load model directly
28
+ ```python
29
  from transformers import AutoTokenizer, AutoModelForCausalLM
30
  tokenizer = AutoTokenizer.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
31
  model = AutoModelForCausalLM.from_pretrained("pfnet/plamo-13b", trust_remote_code=True)
 
39
  top_p=0.95,
40
  temperature=1.0,
41
  )[0]
42
+ generated_text = tokenizer.decode(generated_tokens)
43
+ print(generated_text)
44
  ```
45
 
46
  ## Model Details
47
 
48
  - Model size: 13B
49
  - Trained tokens: 1.5T tokens (English: 1.32T tokens, Japanese: 0.18T tokens)
50
+ - Context length: 4096
51
  - Developed by: Preferred Networkfs, Inc
52
  - Model type: Causal decoder-only
53
  - Language(s): English, Japanese
 
77
  PLaMo-13B is a new technology that carries risks with use. Testing conducted to date has been in English and Japanese, and has not covered, nor could it cover all scenarios. For these reasons, as with all LLMs, PLaMo-13B’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts. Therefore, before deploying any applications of PLaMo-13B, developers should perform safety testing and tuning tailored to their specific applications of the model.
78
 
79
  ## How to cite
80
+ ```tex
81
  @online{PLaMo2023Introducing,
82
  author = {Preferred Networks, Inc},
83
  title = {PLaMo-13B},
 
88
  ```
89
 
90
  ## Citations
91
+ ```tex
92
  @article{touvron2023llama,
93
  title={LLaMA: Open and Efficient Foundation Language Models},
94
  author={Touvron, Hugo and Lavril, Thibaut and Izacard, Gautier and Martinet, Xavier and Lachaux, Marie-Anne and Lacroix, Timoth{\'e}e and Rozi{\`e}re, Baptiste and Goyal, Naman and Hambro, Eric and Azhar, Faisal and Rodriguez, Aurelien and Joulin, Armand and Grave, Edouard and Lample, Guillaume},