Text Generation
Transformers
Safetensors
openelm
custom_code
mahyar-najibi commited on
Commit
25f115c
1 Parent(s): 1a81796

Update README

Browse files
Files changed (2) hide show
  1. README.md +3 -3
  2. generate_openelm.py +5 -0
README.md CHANGED
@@ -20,17 +20,17 @@ We have provided an example function to generate output from OpenELM models load
20
 
21
  You can try the model by running the following command:
22
  ```
23
- python generate_openelm.py --model apple/OpenELM-3B --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2
24
  ```
25
  Please refer to [this link](https://huggingface.co/docs/hub/security-tokens) to obtain your hugging face access token.
26
 
27
  Additional arguments to the hugging face generate function can be passed via `generate_kwargs`. As an example, to speedup the inference, you can try [lookup token speculative generation](https://huggingface.co/docs/transformers/generation_strategies) by passing the `prompt_lookup_num_tokens` argument as follows:
28
  ```
29
- python generate_openelm.py --model apple/OpenELM-3B --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2 prompt_lookup_num_tokens=10
30
  ```
31
  Alternatively, model-wise speculative generation with an [assistive model](https://huggingface.co/blog/assisted-generation) can be also tried by passing a smaller model model through the `assistant_model` argument, for example:
32
  ```
33
- python generate_openelm.py --model apple/OpenELM-3B --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2 --assistant_model apple/OpenELM-270M
34
  ```
35
 
36
 
 
20
 
21
  You can try the model by running the following command:
22
  ```
23
+ python generate_openelm.py --model apple/OpenELM-3B-Instruct --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2
24
  ```
25
  Please refer to [this link](https://huggingface.co/docs/hub/security-tokens) to obtain your hugging face access token.
26
 
27
  Additional arguments to the hugging face generate function can be passed via `generate_kwargs`. As an example, to speedup the inference, you can try [lookup token speculative generation](https://huggingface.co/docs/transformers/generation_strategies) by passing the `prompt_lookup_num_tokens` argument as follows:
28
  ```
29
+ python generate_openelm.py --model apple/OpenELM-3B-Instruct --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2 prompt_lookup_num_tokens=10
30
  ```
31
  Alternatively, model-wise speculative generation with an [assistive model](https://huggingface.co/blog/assisted-generation) can be also tried by passing a smaller model model through the `assistant_model` argument, for example:
32
  ```
33
+ python generate_openelm.py --model apple/OpenELM-3B-Instruct --hf_access_token [HF_ACCESS_TOKEN] --prompt 'Once upon a time there was' --generate_kwargs repetition_penalty=1.2 --assistant_model apple/OpenELM-270M-Instruct
34
  ```
35
 
36
 
generate_openelm.py CHANGED
@@ -1,3 +1,8 @@
 
 
 
 
 
1
  """Module to generate OpenELM output given a model and an input prompt."""
2
  import os
3
  import logging
 
1
+ #
2
+ # For licensing see accompanying LICENSE file.
3
+ # Copyright (C) 2024 Apple Inc. All Rights Reserved.
4
+ #
5
+
6
  """Module to generate OpenELM output given a model and an input prompt."""
7
  import os
8
  import logging