Spaces:

open-llm-leaderboard

/

open_llm_leaderboard

like 11.7k

Running on CPU Upgrade

I just noticed ehartford/dolphin-2.1-mistral-7b has disappeared from the leaderboard.

Was this some accident?

@clem ?

cc @clefourrier @SaylorTwift maybe

Thank you

Hi!
If you need the results urgently, they are still available here, with the full details here.
I'm therefore unsure as to why it does not appear in the leaderboard, but we updated the caching system recently, maybe there's a bug somewhere. We'll look it up!

Thanks for the links!

Not only this, it also appears the original, pretrained, Mistral-7B has dissapeared, all three submissions of it. Also, metrics evolution over time says there's a model with ~76 ARC which can't be found on the leaderboard.

???

It's still not there 😔

Hi @ehartford ! We're investigating as fast as we can, but we are also preparing a big update for the end of the month which is taking most of our time. Thank you for your patience!

Thank you for investigating. Super appreciate it!

Hi @ehartford thanks for your patience ! It is now fixed :)

Thanks You are the best!

again dolphin-2.1-mistral-7b has disappeared

Hi! Have you checked the request file? :)

The request files for both float16 and bfloat16 precision have status=FINISHED and haven't changed in more than 2 weeks according to the Git history, yet they've recently disappeared.
Could this be an issue on your end, perhaps some lingering problem after the big update?

EDIT: they're fine now, must have just been a glitch at time of posting

Are these the two models you are talking about? If yes they are displayed on the leaderboard

It's been fixed by https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard/discussions/377