|
--- |
|
license: apache-2.0 |
|
pipeline_tag: text-generation |
|
tags: |
|
- finetuned |
|
inference: |
|
parameters: |
|
temperature: 0.01 |
|
--- |
|
|
|
A Mistral7B Instruct (https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1) |
|
Finetune using QLoRA on the docs available in https://docs.modular.com/mojo/ |
|
|
|
The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) generative text model using a variety of publicly available conversation datasets. |
|
|
|
## Instruction format |
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
device = "cuda" # the device to load the model onto |
|
|
|
model = AutoModelForCausalLM.from_pretrained("mcysqrd/MODULARMOJO_Mistral-V1") |
|
tokenizer = AutoTokenizer.from_pretrained("mcysqrd/MODULARMOJO_Mistral-V1") |
|
|
|
message = "What can you tell me about mojo roadmap for Scoping and mutability of statement variables?" |
|
|
|
encodeds = tokenizer.apply_chat_template(message, return_tensors="pt") |
|
|
|
model_inputs = encodeds.to(device) |
|
model.to(device) |
|
|
|
generated_ids = model.generate(model_inputs, max_new_tokens=1650, do_sample=True, temperature = 0.01) |
|
decoded = tokenizer.batch_decode(generated_ids) |
|
print(decoded[0]) |
|
``` |
