trust_remote_code procedure

#344
by denizyuret-shallowai - opened

I am working on a new model architecture and would like to submit some results to the leaderboard. I have seen models (phi-1.5, mpt etc) with custom code before but automatic submission is blocked for safety reasons. Is there a procedure I should follow to get my model approved for submission?

Open LLM Leaderboard org

Hi ! We typically allow custom code on the leaderboard only for close collaborators. Unfortunately, we won't be able to run your models for safety reasons. We might in the future sandbox the evaluations to allow for custom code in models and will let you know if that's the case ! :)

SaylorTwift changed discussion status to closed

Sign up or log in to comment