|
--- |
|
license: gpl-3.0 |
|
language: |
|
- en |
|
datasets: |
|
- Mxode/Magpie-Pro-10K-GPT4o-mini |
|
pipeline_tag: text2text-generation |
|
tags: |
|
- text-generation-inference |
|
--- |
|
# NanoLM-25M-Instruct-v1 |
|
|
|
|
|
English | [简体中文](README_zh-CN.md) |
|
|
|
|
|
## Introduction |
|
|
|
In order to explore the potential of small models, I have attempted to build a series of them, which are available in the [NanoLM Collections](https://huggingface.co/collections/Mxode/nanolm-66d6d75b4a69536bca2705b2). |
|
|
|
This is NanoLM-25M-Instruct-v1. The model currently supports **English only**. |
|
|
|
|
|
|
|
## Model Details |
|
|
|
| Nano LMs | Non-emb Params | Arch | Layers | Dim | Heads | Seq Len | |
|
| :----------: | :------------------: | :---: | :----: | :-------: | :---: | :---: | |
|
| **25M** | **15M** | **MistralForCausalLM** | **12** | **312** | **12** | **2K** | |
|
| 70M | 42M | LlamaForCausalLM | 12 | 576 | 9 |2K| |
|
| 0.3B | 180M | Qwen2ForCausalLM | 12 | 896 | 14 |4K| |
|
| 1B | 840M | Qwen2ForCausalLM | 18 | 1536 | 12 |4K| |
|
|
|
|
|
|
|
## How to use |
|
|
|
```python |
|
import torch |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
model_path = 'Mxode/NanoLM-25M-Instruct-v1' |
|
|
|
model = AutoModelForCausalLM.from_pretrained(model_path).to('cuda:0', torch.bfloat16) |
|
tokenizer = AutoTokenizer.from_pretrained(model_path) |
|
|
|
|
|
def get_response(prompt: str, **kwargs): |
|
generation_args = dict( |
|
max_new_tokens = kwargs.pop("max_new_tokens", 512), |
|
do_sample = kwargs.pop("do_sample", True), |
|
temperature = kwargs.pop("temperature", 0.7), |
|
top_p = kwargs.pop("top_p", 0.8), |
|
top_k = kwargs.pop("top_k", 40), |
|
**kwargs |
|
) |
|
|
|
messages = [ |
|
{"role": "system", "content": "You are a helpful assistant."}, |
|
{"role": "user", "content": prompt} |
|
] |
|
text = tokenizer.apply_chat_template( |
|
messages, |
|
tokenize=False, |
|
add_generation_prompt=True |
|
) |
|
model_inputs = tokenizer([text], return_tensors="pt").to(model.device) |
|
|
|
generated_ids = model.generate(model_inputs.input_ids, **generation_args) |
|
generated_ids = [ |
|
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids) |
|
] |
|
|
|
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] |
|
return response |
|
|
|
|
|
prompt1 = "What can you do for me?" |
|
print(get_response(prompt1, do_sample=False)) |
|
|
|
""" |
|
I'm so glad you asked! I'm a large language model, so I don't have personal experiences or emotions, but I can provide information and assist with tasks to help with your tasks. |
|
|
|
Here are some ways I can assist you: |
|
|
|
1. **Answer questions**: I can provide information on a wide range of topics, from science and history to entertainment and culture. |
|
2. **Generate text**: I can create text based on a prompt or topic, and can even help with writing tasks such as proofreading and editing. |
|
3. **Translate text**: I can translate text from one language to another, including popular languages such as Spanish, French, German, Chinese, and many more. |
|
4. **Summarize content**: I can summarize long pieces of text, such as articles or documents, into shorter, more digestible versions. |
|
5. **Offer suggestions**: I can provide suggestions for things like gift ideas, travel destinations, books, or movies. |
|
6. **Chat and converse**: I can engage in natural-sounding conversations, using context and understanding to respond to questions and statements. |
|
7. **Play games**: I can play simple text-based games, such as 20 Questions, Hangman, or Word Jumble. |
|
8. **Provide definitions**: I can define words and phrases, explaining their meanings and usage. |
|
9. **Offer suggestions**: I can provide suggestions for things like gift ideas, travel destinations, or books to read. |
|
10. **Entertain**: I can engage in fun conversations, tell jokes, and even create simple games or puzzles. |
|
|
|
Which of these methods would you like to do? |
|
""" |
|
``` |
