Background: I ran into numerous issues when presenting and running this King of the Hill challenge back in October: Write a bot to play Grid Game - I'm cooking up a new, similar challenge, but taking things a bit more slowly and preparing more thoroughly to try and pre-emptively solve the issues we hit then.
So this was the intended path for entries to the Grid Game competion:
- Someone works on a bot, likes thier solution.
- It's added to a Github Gist.
- They past the gist and the code to an SO answer.
- My code grabs the gist via the Github gist api.
- The tournament is run with the latest version of the gists.
I also added a note that entrants could choose whether or not to consent to their entry becoming a part of the code base.
Here are some issues we found with that methodology:
- The Gist API rate-limits get calls, so I had to cache the code to avoid being locked out.
- Entries were in multiple places, causing confusion on whether the latest version was used:
- SE answer
- Cached in local memory
- Cached in the code base
Long story short, this was a real headache when running the tournament to get results. A lot of in-line changes had to be made to try and correct it.
What I'd like to do about this, is to make the entry a Pull Request in Github, making the repository a single, central source.
It would look like this:
- The entry is a JS class, in a certain folder in the application.
- There's a validator script which ensures it follows the rules of the challenge and doesn't break the applicaiton.
- The result is submitted as a pull request.
- If it passes the validator, it's accepted in to the repo and run as part of the tournament.
- Updates to an entry would be made in the same way, modifiying the existing file.
- A 'single source of truth' for entries.
- Everyone can train their entries on all current entries.
- Less in-line problem solving.
- To enter the contest would make the entrant a contributor to an open-source repo.
- The SE answer would probably be quite short, just a stub pointing to the pull request or file in the repo?
Is it acceptable to use Github Pull Requests as entries to a code challenge?
What other issues might this methodology introduce?
Is there a better, regularly used way to import a few mid-length entries (probably between 10 and 100 lines) into an application to run a KotH challenge which I haven't considered?