|
--- |
|
model-index: |
|
- name: abacaj/mistral-7b-sft |
|
results: |
|
- task: |
|
type: text-generation |
|
dataset: |
|
type: openai_humaneval |
|
name: HumanEval |
|
metrics: |
|
- name: pass@1 |
|
type: pass@1 |
|
value: 54.27 |
|
verified: false |
|
- task: |
|
type: text-generation |
|
dataset: |
|
type: mbpp |
|
name: MBPP |
|
metrics: |
|
- name: pass@1 |
|
type: pass@1 |
|
value: 38.00 |
|
verified: false |
|
- task: |
|
type: text-generation |
|
dataset: |
|
type: mmlu |
|
name: MMLU |
|
metrics: |
|
- name: pass@1 |
|
type: pass@1 |
|
value: 45.89 |
|
verified: false |
|
language: |
|
- en |
|
--- |
|
|
|
How to run inference: |
|
```python |
|
import transformers |
|
import torch |
|
|
|
|
|
def fmt_prompt(prompt: str) -> str: |
|
return f"""[Instructions]:\n{prompt}\n\n[Response]:""" |
|
|
|
|
|
if __name__ == "__main__": |
|
model_name = "abacaj/mistral-7b-sft" |
|
tokenizer = transformers.AutoTokenizer.from_pretrained(model_name) |
|
|
|
model = ( |
|
transformers.AutoModelForCausalLM.from_pretrained( |
|
model_name, |
|
) |
|
.to("cuda:0") |
|
.eval() |
|
) |
|
|
|
prompt = "If A is greater than B and B is greater than C does that make A greater than C?" |
|
prompt_input = fmt_prompt(prompt) |
|
inputs = tokenizer(prompt_input, return_tensors="pt").to(model.device) |
|
input_ids_cutoff = inputs.input_ids.size(dim=1) |
|
|
|
with torch.no_grad(): |
|
generated_ids = model.generate( |
|
**inputs, |
|
use_cache=True, |
|
max_new_tokens=512, |
|
temperature=0.2, |
|
top_p=0.95, |
|
do_sample=True, |
|
eos_token_id=tokenizer.eos_token_id, |
|
pad_token_id=tokenizer.pad_token_id, |
|
) |
|
|
|
completion = tokenizer.decode( |
|
generated_ids[0][input_ids_cutoff:], |
|
skip_special_tokens=True, |
|
) |
|
|
|
print(completion) |
|
``` |
|
|
|
Evals: |
|
|
|
 |
|
|
|
 |
|
|
|
Code to train model: |
|
https://github.com/abacaj/train-with-fsdp |
