Spaces:

latticeflow

/

compl-ai-board

like 15

Running

Hi @djstrong ,

we are currently addressing them one by one, in the order we received them. At this point, we are evaluating Gemini models. We are changing how the submissions works to make this more transparent.

If you have a model in mind, feel free to also write here and we'll try to make an estimate.

Best,
Pavol

Thank you! We have developed Polish-English model: https://huggingface.co/speakleash/Bielik-11B-v2.3-Instruct and working on the next version. We are interested how it compares to Mistral 7B (which is a base for this model) and how we can use this benchmark during development of new versions of our models.

I see, that would be nice to compare indeed. Currently the evaluation benchmarks are done in English, would this be fine or are you looking for something in Polish only (given this is how the model is finetuned).

For English, we should be able to start the eval this week. We'll keep you posted.

Thank you, English is fine, the model is trained on Polish and English.

Of course, it would be nice to have similar benchmark in Polish - maybe we (SpeakLeash team) can cooperate on this?