11
\$\begingroup\$

In this question, every "game" takes about 5 seconds. Ten thousand runs, maybe eight thousand, are planned. Running these on a single computer would take forever, so I suggested letting competitors and volunteers run some of the games. However, this result in cheating. My question is: should this be allowed?

\$\endgroup\$
6
  • \$\begingroup\$ I don't know if this is a question meta can answer as in general, but in this case, another option is to offload the work to a free cloud server - for eg. koding.com and MS Azure both seem to have free trials (though both require credit card details for verification - and there may be other options that I don't know about). Ten thousand runs of 5 seconds comes out to about 14 hours (possibly less on the server machine), you can just let it run there and check back for results. \$\endgroup\$
    – Sundar R
    Commented Aug 12, 2018 at 20:27
  • \$\begingroup\$ Well, in general, it would be nice to know. Plus, as bots get more complex, that could easily escalate to 15s per round \$\endgroup\$
    – rydwolf
    Commented Aug 13, 2018 at 0:09
  • 5
    \$\begingroup\$ My KotHs typically run anywhere from 2 to 24 hours. Yeah, you can get help, but personally, I simply see it as a time commitment that I need to give when I post a KotH. (I also like to keep current scoreboards, so my computer is basically constantly running while a KotH is going) \$\endgroup\$
    – Nathan Merrill
    Commented Aug 13, 2018 at 6:23
  • \$\begingroup\$ Just reduce it to about 500 runs.... or limit runtime of bots. \$\endgroup\$
    – DELETE_ME
    Commented Aug 13, 2018 at 6:58
  • \$\begingroup\$ Or tell everybody to optimize their bots first \$\endgroup\$
    – rydwolf
    Commented Aug 13, 2018 at 13:45
  • \$\begingroup\$ Even if the bots are what are taking up most of the time, it may sometimes still be possible for the KotH author to speed things up to some extent. For example, if there is a limited set of inputs and bots are required to be deterministic, then outputs may be cached. \$\endgroup\$
    – trichoplax is on Codidact now
    Commented Aug 19, 2018 at 22:23

1 Answer 1

Reset to default
10
\$\begingroup\$

Yes, but not competitively

The controller should be made available by the person running the KotH, to allow people to test and experiment, however they should not have a part in the final run that determines the winner. It should be up to the original poster to plan ahead of time how they intend to run the game, and if their game takes a long time to run, they should either optimize their controller, or have thought about that before hand (and made a less complex game).

One solution would be to have a cutoff date for entries, and at that point begin running the controller with all available entries, perhaps on a server somewhere, so that it can be left running as long as it needs. Then once the controller has finished, results can be posted.

\$\endgroup\$
3
  • \$\begingroup\$ The bots are what needs optimization...There're at least 30 at the moment, and nearly all of them have at least 2 for loops \$\endgroup\$
    – rydwolf
    Commented Aug 13, 2018 at 13:44
  • 3
    \$\begingroup\$ @RedwolfPrograms That is a concern that should have been taken into consideration before posting the KotH. As I said, I believe it's the poster's responsibility to ensure they have the means to run the KotH properly before posting, hence why I have no qualms with KotHs requiring certain languages (or at least stating that the language choice must include basic instructions on how to run it on [insert machine the poster intends to run the KotH on]). The poster could ask for participants to optimize their entries, especially if optimizations are easily spotted, but should not expect they do so \$\endgroup\$
    – Mayube
    Commented Aug 13, 2018 at 14:02
  • 4
    \$\begingroup\$ @RedwolfPrograms there are a couple of things I've seen effective at providing incentive for fast bots: 1. Provide a "helper" API the bots can use for common operations that are fast. People also copy/paste bots, so having efficient "starter bots" are also useful. 2. Measuring bot response time. This is commonly said as "Your bot must respond in X ms", but I think it is far more effective to say "Your bot must have an average response time of X ms measured over Y turns" \$\endgroup\$
    – Nathan Merrill
    Commented Aug 13, 2018 at 14:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .