8
\$\begingroup\$

A challenge must have an objective winning criterion. However, does this have to define a single winner, or is it acceptable for a challenge to have categories of winners? We already encourage people to think in terms of competing with other solutions in the same language, and we often include a leaderboard that shows winners by language.

A recent example is this challenge to write code that censors its own 4 letter words by replacing the middle 2 letters, while still maintaining the same behaviour when run. The winning criterion is code golf, and it is suggested that solutions compete with other solutions that contain the same number of 4 letter words. An automated leaderboard can easily be constructed for this, and it allows competition with oneself and others in the way that any other winning criterion does.

It seems analogous to sports having weight categories.

Do we want this type of multi-category challenge to be on topic? If so what guidelines should we have for making this work well?

\$\endgroup\$
5
  • 2
    \$\begingroup\$ I've considered splitting some of my challenges into two categories, such as regex vs no regex, to allow a technique while not shutting out interesting alternatives. \$\endgroup\$
    – xnor
    Apr 27, 2016 at 8:52
  • 2
    \$\begingroup\$ I think this is fine (mostly) in theory, but I think there are several practical problems. In the example you gave where submissions were split into categories by the number of words, I feel like I would have to close it as "too broad". Wouldn't there be an infinite number of reasonably competitive answers in (almost) every language? It would also seem odd to have them in the same place, as it would imply that they are comparable (even if the challenge says they shouldn't be). xnor's split seems much better, but it might run into problems of "what is regex" and risks being too unclear IMO. \$\endgroup\$
    – FryAmTheEggman
    Apr 27, 2016 at 15:49
  • \$\begingroup\$ @FryAmTheEggman I think the requirement that none of the censored words be redundant should keep the number of words replaced quite low. \$\endgroup\$
    – trichoplax is on Codidact now
    Apr 27, 2016 at 16:09
  • \$\begingroup\$ Perhaps the example I gave is not the best way to judge this question. I included it because it was the one that prompted me to think about this, but perhaps we need a different example challenge to show the good and bad points of this idea. \$\endgroup\$
    – trichoplax is on Codidact now
    Apr 27, 2016 at 16:10
  • \$\begingroup\$ Now that the example challenge has been deleted (not necessarily for the same reason as is being discussed here), we really could do with another example... \$\endgroup\$
    – trichoplax is on Codidact now
    Apr 27, 2016 at 16:12

2 Answers 2

Reset to default
4
\$\begingroup\$

This is a good thing provided the challenges belong in the same post

If the challenge is "Output the sum of N numbers, or alternatively their product" that just clutters the solutions of two distinct challenges together. They should be separate challenges in separate posts.

However, where the categories are inherently part of the same challenge, such as the example in the question, I favour the idea of a winner per category over trying to artificially force a universal scoring mechanism.

A contrived score based on bytes modified by category means that getting the best score requires testing all the categories, giving the same explosion of extra work as having bonuses. It also means that in many cases some languages will not fit well with the scoring method, resulting in only one category being worth trying for that language.

The only problem I anticipate with having a winner per category is that there is only one acceptance tick. Personally I don't see that as a problem at all, as I am strongly in favour of PPCG not having acceptance ticks

\$\endgroup\$
2
\$\begingroup\$

A very emphatic "no"

This suggestion is a conglomeration of chameleon challenges (because the real challenge would not be solving the challenge, but picking the optimal category), bonuses in code golf (because the large number of sub-challenges to solve would dwarf the actual challenge, like with bonuses), and multi-part challenges (because each category would essentially be a challenge in of itself). As a result, it would be nigh-impossible to do well.

If people want to informally compare answers ("Yay, my submission has the best score out of all the submissions which use X strategy!"), that's fine. Allowing formal categories in challenges opens up many cans of worms.

As it stands, we require our challenges to have an objective winning criterion that selects exactly one submission as the winner. Introducing categories into challenges would cause them to have multiple winners. We need to have a discussion about whether or not we want to be able to have multiple winners before we can even consider allowing categories.

\$\endgroup\$
4
  • 1
    \$\begingroup\$ I agree with chameleon challenges, bonuses and multi-part challenges being a problem, but I see this differently. I see categories as giving parallel challenges, rather than having to try them all to find one overall best approach. The point is that the winner of the 2 word category wouldn't be better or worse than the winner of the 3 word category. There would be no need to try all of them, just to compete to be the best in the category you choose, just like we compete to be the best in a chosen language. \$\endgroup\$
    – trichoplax is on Codidact now
    Apr 27, 2016 at 8:53
  • \$\begingroup\$ @trichoplax In essence, it's posing two (or more) different challenges within the same challenge, which makes it fall into multi-part challenge territory (the challenge would be "solve one of these sub-challenges, best score in each category wins the category", which would be the bad kind of multi-part challenge). Since there is only one overall winner, people will be picking the category which they can optimize best for, which causes the same problems as bonuses. \$\endgroup\$
    – user45941
    Apr 27, 2016 at 8:56
  • 2
    \$\begingroup\$ @Mego I think trichoplax has a point. "Choosing the category which they can optimize best for" in this case is no different than "Finding a language which they can optimize best for", if we really consider this analogous to looking at per-language winners in standard code golf. That said, I haven't really settled on my own opinion on this matter yet... it's a bit of a slippery slope towards abandoning objective winning criteria, while at the same time I do prefer per-language comparisons. \$\endgroup\$
    – Martin Ender
    Apr 27, 2016 at 9:00
  • 1
    \$\begingroup\$ @MartinBüttner I also like the per-language comparisons, but they're done informally. Formally, challenges have an objective winning criterion that select one overall winner. Having categories makes it impossible to choose one winner overall. \$\endgroup\$
    – user45941
    Apr 27, 2016 at 9:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .