xtuner

/

llava-llama-3-8b-v1_1

like 120

Follow

xtuner 66

@RDson I appreciate that you did this, but since the model is based on Llava 1.5 it will require a mmproj to deliver vision capabilities, right?

You are correct, I completely forgot about that 🤦

No worries, I have attempted to create an mmproj and run up against issues, I have posted here with more information: https://huggingface.co/xtuner/llava-llama-3-8b-v1_1/discussions/3

I'll keep the files up in case anyone for whatever reason needs them

@jeiku @RDson Hi!
We are trying to convert the current model format to the official format of LLaVA, and it is expected to be completed in 1-2 days.

@jeiku @RDson
We have released weights similar to LLaVA v1.5/v1.6 architecture here. Welcome to try this model with your workflow!

How to run this with Ollama?

Hi @RDson @jeiku
GGUF model is here.

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf

Hi, On ollama I encountered this problem: when I make it interpret the first image everything is ok, the model works correctly, but if I make it interpret a second image it starts interpreting the previous image, how can I overcome this problem?

How to run this with Ollama?

simply create a new modelfile for ollama,
ollama's documentation explains how to do it, it's not complicated..
ollama create llm_name -f modelfile.txt