mistral-community/pixtral-12b-240910 · How to use safetensors?

Sep 22

Hey I have downloaded.safetensors and .json files. Now I want a python code to load this . safetensors file and run pixtral model. Can anyone say how to load . safetensors in python? I want to give a prompt to this model.

ArthurZ

Unofficial Mistral Community org Sep 22

Would recommend you this https://huggingface.co/mistral-community/pixtral-12b with:

from PIL import Image
from transformers import AutoProcessor, LlavaForConditionalGeneration
model_id = "mistral-community/pixtral-12b"
model = LlavaForConditionalGeneration.from_pretrained(model_id)
processor = AutoProcessor.from_pretrained(model_id)

IMG_URLS = [
"https://picsum.photos/id/237/400/300", 
"https://picsum.photos/id/231/200/300", 
"https://picsum.photos/id/27/500/500",
"https://picsum.photos/id/17/150/600",
]
PROMPT = "<s>[INST]Describe the images.\n[IMG][IMG][IMG][IMG][/INST]"

inputs = processor(text=PROMPT, images=IMG_URLS, return_tensors="pt").to("cuda")
generate_ids = model.generate(**inputs, max_new_tokens=500)
ouptut = processor.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

prathi1729

Sep 22

U are not using .safetensor file or json file in this code. I have downloaded these files in local. I want to load these files in python and then use it.