The score is fake.
The best score of this case is said to be 500 from https://colab.research.google.com/github/araffin/tools-for-robotic-rl-icra2022/blob/main/notebooks/optuna_lab.ipynb. I don't understand why so many people got 1000. After checking the evaluation code and found the score is reported in the code controlled by myself. I think it would be better to be evaluated by Hugging Face system.
Just leave it here as a proof.
Agree, The same is true in Taxi where a lot of people have manipulated the seed to get better scores. I think @ThomasSimonini will have to do a bit of a re-vamp at some point to try and make this more difficult for people to achieve.
Also, all the boards that are dominated by perfect scores like frozenlake4*4-noslippery should probably be removed as they do not demonstrate anything very significant.
I would like to see some Doom and MicroRTS boards 😁
Hey there. Since it's for educational purposes we decided when we created the leaderboard to not check the scores. If people want to cheat (idk why because there's nothing to win) they can and we don't plan for now to put a checking system (again because it's educational purposes).
Which doom environement do you want me to add ? I can do it 😁
Ooh. How about defend_the_centre and defend_the_line for starters and maybe add in doom_battle? 😆
You should probably add health_gathering_supreme as well because it is the one in the tutorial 😊. The doom environments are very fun and are really good for demonstrating the value of sample factory.
Love it @ThomasSimonini
Hey there. Since it's for educational purposes we decided when we created the leaderboard to not check the scores. If people want to cheat (idk why because there's nothing to win) they can and we don't plan for now to put a checking system (again because it's educational purposes).
Which doom environement do you want me to add ? I can do it 😁
Hi @ThomasSimonini Understood. Usually, we believe everything here. The downside of the fake score is confusing me how I can do better because no matter how many times I tried, I still can't get close to those good looking scores. That is why I put this proof here to let others know the truth.