Post
🔥 New LLM leaderboard on the hub: NPHardEval!
It uses questions of logic, of different mathematical complexities, as a proxy for reasoning abilities. It notably removes questions relying on arithmetic, to really focus on logical abilities.
What's interesting imo is the potential to really study a model performance at different levels of complexity.
Bonus: Since the questions can be generated automatically, it's going to be dynamic, updated monthly! 🚀
NPHardEval/NPHardEval-leaderboard
Read more about how their questions are generated in the intro blog: https://huggingface.co/blog/leaderboards-on-the-hub-nphardeval
Congrats to @lizhouf , @wenyueH , @hyfrankl and their teams!
It uses questions of logic, of different mathematical complexities, as a proxy for reasoning abilities. It notably removes questions relying on arithmetic, to really focus on logical abilities.
What's interesting imo is the potential to really study a model performance at different levels of complexity.
Bonus: Since the questions can be generated automatically, it's going to be dynamic, updated monthly! 🚀
NPHardEval/NPHardEval-leaderboard
Read more about how their questions are generated in the intro blog: https://huggingface.co/blog/leaderboards-on-the-hub-nphardeval
Congrats to @lizhouf , @wenyueH , @hyfrankl and their teams!