Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.7k

Running on CPU Upgrade

I keep getting an error: Model "migtissera/SynthIA-7B-v1.3" was not found on hub!

Yeah I was getting this for Mistral models, I think this don't support the Mistral architecture?

Hi ! This is weird as your model is available on the hub. Can you give me all the specification you try to submit your model with ? (model type, precision, weight type, etc..)

Hey,
model type: fine-tuned
Float16
Main branch
Weights - original

I'm having the same issue with bhenrym14/mistral-7b-platypus-fp16

Instruction-tuned
float16
main branch
original

Yeah, couple of other users said the same thing.. Could be Mistral specific

Same issue here too, did anyone used Axolotl to do the full FT , also did anyone use bf16 weights for submission to
Hub, Just trying to find out some common pattern?
Also, IMHO if HF team were able to able to conduct evals on original mistral pretrained based and instruct model then I guess there should not be issue with Mistral Architecture based Fine Tuned models for eval submissions.

Pankaj! Awesome to see you back here :)

Pankaj! Awesome to see you back here :)

you guys keeping the lights on great stuff. More and more I am founding myself enjoying talking to SynthIA :)

Haha.. Man your orca-mini model is awesome! Looking forward to the Mistral fine-tune.. Did you do any evals?

@ehartford said he's getting the same error

Getting same issue with https://huggingface.co/SkunkworksAI/Mistralic-7B-1

ehartford/dolphin-2.0-mistral-7b too

Seems to be an issue with Mistral fine-tunes.. Any update @SaylorTwift ?

It looks like it is related to all mistral models, will look into it today :)

Hi everyone ! Good news, I just fixed the model submissions. For those who are curious, mistral uses the nightly branch of transformer, updating the requirements fixed it. :)
You can now submit your fine-tuned mistral models.
@ehartford and @migtissera I submitted your models already so if you try and see that you cannot do it, it's normal :)

Awesome thank you 🙏 @SaylorTwift i confirmed I can submit my model now to the LB.