As a bajillion people noted, allowing people to check the scores of the test set allows people to cheat by unfairly tuning their algorithm to improve their score. Our hope in providing public scores was to inspire a little friendly competition end motivate people to get the most out of their algorithms...
To correct this flaw in our contest design, we are still going to use the "test set" to update the leaderboard once each day. Think of the current test set as a validation set. There will be a "final test set" released on 19 May 2008. Algorithms will be scored on the basis of performance on the final test set only. Scores will not be released for the final test set until after the competition has closed.
Subscribe to:
Post Comments (Atom)
1 comment:
How do we submit our predictions? the "click here" link is not active.
Post a Comment