metadata
base_model: unsloth/llama-3-8b-bnb-4bit
language:
- en
license: apache-2.0
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
Uploaded model
- Developed by: jjsprockel
- License: apache-2.0
- Finetuned from model : unsloth/llama-3-8b-bnb-4bit
C贸digo para descarga: El siguiente es el c贸digo sugerido para descargar el modelo usando Unslot:
import torch
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "jjsprockel/Patologia_lora_model1", # YOUR MODEL YOU USED FOR TRAINING
max_seq_length = 2048, # Choose any! Llama 3 is up to 8k
dtype = None,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{}
### Input:
{}
### Response:
{}"""
C贸digo para la inferencia:
Get the instruction from the console
El siguiente codigo demuestra como se puede llevar a cabo la inferencia.
instruction = input("Ingresa la pregunta que tengas de Patolog铆a: ")
inputs = tokenizer(
[
alpaca_prompt.format(
instruction, # instruction
"", # input
"", # output - leave this blank for generation!
)
], return_tensors = "pt").to("cuda")
from transformers import TextStreamer
text_streamer = TextStreamer(tokenizer)
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 2048)
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
