18
\$\begingroup\$

We occasionally have challenges which consist of multiple separate subchallenges, whose individual scores are simply combined at the end. These are often known as "golf courses". While these are often quite popular they do have some problems. Mainly it's not clear how we should deal with people copying one of the subchallenges from another answer, and how these should be treated in the context of partial duplicates.

Therefore: are these actually a good fit for our community, should we change the way we run these contests or should they be outright disallowed?

We might want to distinguish between multi-part challenges where the individual parts strongly affect each other and challenges where they are mostly orthogonal.

\$\endgroup\$
23
\$\begingroup\$

As I said in the question, I think we need to distinguish several cases here. (This post got a bit long. There's a summary at the bottom, but if you disagree with it, I'd appreciate if you took the time to read my arguments as well and bring up your position in comments or a separate answer.)

Subchallenges without any interaction

These are the plain "golf courses". They pose a number of (usually small) challenges which have to be solved and the final result is the sum of all individual scores. I think these should be strictly disallowed, because they have some severe problems (touched upon in the question):

  • They are essentially immune to being closed as duplicates, because you could theoretically mash up any N challenges we've done before and who's to say it's a duplicate of any of them? What if you mash up N-1 and add one new challenge? What if there are two such challenges that overlap in half their subchallenges? Despite not being a Q&A site, PPCG is still built on Q&A software, and one of the main principle on other SEs is that one post should ask a single question. Since these multi-part challenges don't follow this principle they circumvent some of the quality control mechanism that are in place on SE.
  • They end up with people copying better parts from other answers, so it's much harder to determine a winner. And it's hard to prevent this, because eventually you might glance at another answer and see a much better approach, and the only alternative to using it is locking in that part of your answer, because all your own approaches will probably be strongly influenced by it. This also means, someone can just pick the best parts from all existing answers and make a new winning answer. Probably unpopular, but in no way illegal with our current policies.
  • The main question is, why didn't you ask these as N separate challenges in the first place? I can think of three possible answers to that:
    1. "We'd already done some of them before." See the first bullet point.
    2. "These are too trivial to be interesting on their own." But does throwing together several trivial problems make them any less trivial and more interesting? Posting small and very simple challenges isn't necessarily bad as long as they still leave some room for creativity (at least in harder-to-use languages). If a subchallenge is too trivial even for that, putting it together with several others isn't going to improve it.
    3. "These challenges are closely related and follow a common theme." But that doesn't mean you can't split them up into several posts, such that answers to each subchallenge can still receive independent votes, the challenges can be put on hold if necessary and they can be used as dupe targets for other challenges. You can still create an overall leaderboard and maybe reward the overall winner with a bounty. Examples of this type of multi-challenge series are Optimizer's ASCII Art of the Day and my own Random Golf of the Day. The first challenge in the series can act as a hub to the rest, so that someone interested in it, can still easily find all the parts.

Subchallenges with little interaction

This is actually the most common form of multi-part challenge. The very first 9 Hole Challenge penalised reuse of languages, Dennis's First code golf decathlon prohibited it entirely and gave out points for being the best for each task or language, Mego's The Interview: The Front Nine required all answers to be in the same language. I don't think this is sufficient to warrant squeezing all the subchallenges into a single post. It still has the first two problems from above. While you can't copy any other answer, once enough people have answered, someone could still come along and reduce the challenge to an optimisation problem of which existing subanswers to combine for the best overall score. Moreover if you want to determine an overall winner based on subchallenge winners and per-language winners you can still do that with a custom leaderboard and multiple challenge posts.

The main point is that the total set of answers you'll get for the subchallenges isn't in any way different from the set of answers you'd get if these were asked separately. Therefore, they don't add any value for the content of the site (when compared with posting them separately). These should still be disallowed.

I'd also put Calvin's Hobbies' Cramming The Gramming - Twelve Task Tweet in this category since the optimal solution is still to golf each subtasks individually and then simply count how many of them fit into the byte limit 140.

Subchallenges with strong interaction

These are challenges where the solutions the individual parts are strongly linked. For example Dennis's Overcoming cluster size asked for a polyglot within a size limit that solved as many tasks as possible. jimmy23013's Print strings without sharing characters disallowed reusing characters between multiple parts (I feel like we've had a similar challenge with a finite set of tasks instead of trying to get as many words as possible, but I can't find it). This would also include challenges where substantial amounts of code can be reused between multiple subtasks (e.g. because it's one program solving N closely related tasks).

I think these are fine because the separate solutions strongly affect each other. You can't just solve each part individually, nor can you combine the best sub-solutions from existing answers, because they likely won't fit together under the given rules. Separating them out into separate challenges would either be impossible or completely change the answers you get.

They still have one problem though: the concept of duplicates. I can ask such a challenge with concept X and N challenges. Now the next person can ask another such challenge with the same concept but a different set of challenges. That may seem fine, but again what do we do if half the subchallenges are the same? Do we really want every possible multi-part restriction applied to every possible subset of challenges? In these cases, we can probably assume that it's concept X that makes the challenge, not the N subtasks. While the specific tasks will of course change the actual solutions, and maybe also to some degree the nature of how subsolutions are combined, the core of the challenge will always be the restriction that links the subparts. Therefore, these challenges should be judged on their overall concept, not the particular subtasks chosen. When another challenge gets asked with the same concept but different challenges, I'd argue that this different set of challenges would have to significantly affect how the challenge can be approached on a large scale, otherwise they should be closed as duplicates.

In Summary

  • Combining multiple tasks into a challenge without any interaction should be disallowed. If the subtasks are actually interesting, ask them separately.
  • If there is only small interaction, mainly related to restrictions, penalties or bonuses for the choice of languages, these still have the same problems and should be disallowed.
  • If there is strong interaction between them, these could actually add something interesting to the site and should be allowed. However, duplicates should be determined based on the nature of the interaction not of the chosen subchallenges, unless these actually affect possible approaches to the challenge at large.

The distinction between the last two cases is of course not always clear cut and will require human judgement. As rules of thumb we can ask the following questions:

  • Is it optimal to solve each subtask individually and combine them into one answer? (Potentially subject to language restrictions.)
  • Can someone come along (after enough answers have been posted to make language restrictions irrelevant) and mix and match subtasks from existing answers to obtain a competitive score?
  • Would the set of answers for the subtasks be any different if these subtasks were posted as separate challenges?
\$\endgroup\$
  • 6
    \$\begingroup\$ The defining line should be: Does the difficulty of the challenge consist of combining the different parts? IMO. \$\endgroup\$ – Nathan Merrill Feb 16 '16 at 23:12
  • \$\begingroup\$ Is it OK to create a challenge about other challenges? \$\endgroup\$ – dorukayhan Jun 15 '16 at 18:29
  • \$\begingroup\$ @dorukayhan I'm not sure what you mean and it seems a bit unrelated to the discussion in this post. Maybe we can discuss this in chat? \$\endgroup\$ – Martin Ender Jun 15 '16 at 18:32
0
\$\begingroup\$

Multi-part challenges aren't ideal, but shouldn't be forbidden

To be fair, "multi-part" is a relative term. A challenge that asks for one program that takes the square root of the cosine of a number is not all that different from a multi-part challenge that asks for one program that takes the cosine of a number and another program that takes the square root of a number.

Multi-part challenges aren't that common anyway (and don't seem to be getting any more or less common), so a few won't tarnish the site too much.

The primary complaints about them mentioned in the question were:

(1) Mainly it's not clear how we should deal with people copying one of the subchallenges from another answer, and (2) how these should be treated in the context of partial duplicates.

I'd argue that all challenges share issue (1) - copying sections of code or algorithm ideas can happen anywhere. It's easier in multi-part challenges but still, whether it's allowed or not is a different meta question.

Issue (2) is the real crux in my opinion, since we put so much effort into our avoidance of duplicates. I'd say that if no part of a multi-part question is a duplicate, why not allow it? The author still came up with the ideas themselves and why not let them combine them into one challenge if they want to?

If some parts are duplicates then that's a different story. I'm not really sure what the best solution is but here are a few ideas (maybe this could be its own meta question):

  • Not allow any duplicate parts. (Encourage author to quickly change/remove offending parts.)
  • Allow it if the majority of the parts are non-duplicates.
  • Don't have a hard rule and let people vote to close as duplicate naturally. (Less official but perhaps more fair in some cases since it's kinda case by case.)

(Recent example of a multi-part question, just for some context.)

\$\endgroup\$
  • 6
    \$\begingroup\$ I disagree. Why should we allow things that are confusing and hard to handle, when either the separate parts of it would be good enough to post on their own? If the separate parts aren't good on their own, the total question isn't good either, in my opinion. Basically I'd say the real question is why should we allow it, it doesn't seem to add anything other than headaches to the site that separate posts wouldn't. (Although note that if the challenges heavily depend on each other, I'd say that's a different story) \$\endgroup\$ – FryAmTheEggman Apr 20 '16 at 16:12
  • 2
    \$\begingroup\$ This doesn't address the issue I have with multi-part challenges (From Martin's answer) "The main question is, why didn't you ask these as N separate challenges in the first place?". Sure, it conceivably might not hurt the challenge, but will it really ever help the challenge? \$\endgroup\$ – Morgan Thrapp Apr 20 '16 at 16:16
  • \$\begingroup\$ @MorganThrapp (and Fry) It might not help the challenge but I'd say that's a reason to downvote it, not forbid it. \$\endgroup\$ – Calvin's Hobbies Apr 20 '16 at 16:23
  • 4
    \$\begingroup\$ I'd rather see it closed and the question writer given the chance to break it into multiple challenges. I think having a policy that says we disallow multi-part challenges makes it easier for us to point the writer at something concrete. \$\endgroup\$ – Morgan Thrapp Apr 20 '16 at 16:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .