Over on Puzzling.SE, the community there has been discussing various ways to reward puzzle authors for creating exemplary puzzles. Given that we're like Puzzling.SE in that we're not a Q&A site, and that we've discussed increasing reputation gain/loss for challenge upvotes and downvotes, it seems fitting that some of the ideas discussed over at Puzzling.SE might also work well for our site.

The Problem

Creating a good challenge is much harder than writing a solution to a challenge. Challenges often spend several days or longer in the Sandbox, where authors receive feedback to smooth out rough edges and ensure that their challenges are as high-quality as possible. In spite of this, challenges are treated the same as questions on any other Q&A site on the network: 5 reputation added per upvote, 2 reputation subtracted per downvote, and no system to reward awesome challenges. Good solutions are rewarded much more (double the rep, with the possibility for bounties), which gives the appearance that we value solutions more than challenges (which is very false - a good challenge is just as valuable as a good solution, if not more so).

What changes could be made to reward users who create exemplary challenges (other than doubling the reputation gain/loss from votes, which is the subject of an existing feature request)?

  • 9
    \$\begingroup\$ "Creating a good challenge is much harder than writing a solution to a challenge" Generally that's true but it's not always the case :) \$\endgroup\$
    – Fatalize
    Commented May 23, 2016 at 12:15
  • \$\begingroup\$ Pretty much every (highly voted) suggestion in the puzzling.SE meta question I think would apply well here \$\endgroup\$
    – Blue
    Commented May 23, 2016 at 15:13
  • 3
    \$\begingroup\$ One thing I will say that we are already doing right is how freely the upvotes flow on this site. Even my worst challenge here has more votes than my best question on SO. That's a great thing, and it's even the initial thing that attracted me to this site. So, yeah! Keep it up! \$\endgroup\$
    – DJMcMayhem
    Commented May 23, 2016 at 16:11
  • 1
    \$\begingroup\$ Also note that the rep gain change for questions was pretty much vetoed by @GraceNote. \$\endgroup\$
    – Riker
    Commented May 23, 2016 at 19:33
  • \$\begingroup\$ @EᴀsᴛᴇʀʟʏIʀᴋ I don't remember that - source? \$\endgroup\$
    – user45941
    Commented May 23, 2016 at 20:23
  • 1
    \$\begingroup\$ @Mego chat.stackexchange.com/transcript/240?m=27434068#27434068 \$\endgroup\$
    – DJMcMayhem
    Commented May 23, 2016 at 20:32
  • 1
    \$\begingroup\$ @DrGreenEggsandHamDJ Mileage may vary - my highest voted challenge here has fewer votes than my only question on SO. I agree we generally do well at upvoting good challenges though. \$\endgroup\$ Commented May 23, 2016 at 20:43
  • 5
    \$\begingroup\$ @trichoplax A significant number of our highest-voted questions are very trivial (2014, Showcase, Hello, World!, Print every character your program doesn't have, Olympic Rings... the list goes on). In my personal experience, trivial challenges get more votes than challenging challenges. \$\endgroup\$
    – user45941
    Commented May 23, 2016 at 20:57
  • \$\begingroup\$ @Mego yes sadly true. Despite the massively upvoted trivial challenges, there still seem to be a lot of more substantial challenges getting upvotes though. \$\endgroup\$ Commented May 23, 2016 at 21:04
  • 1
    \$\begingroup\$ I am keen to see more ways to encourage/reward challenge writers though. And not just because I am more inclined to challenge writing than solution writing... \$\endgroup\$ Commented May 23, 2016 at 21:04
  • \$\begingroup\$ The proposed challenge tour will reward a fixed selection of challenges, but doesn't provide a way of rewarding new challenges after that. Hopefully suggestions here can apply to future challenges without a cut off. \$\endgroup\$ Commented May 23, 2016 at 21:12
  • \$\begingroup\$ Most of the suggestions in the puzzling.SE discussion seem to require a change to the Q&A engine. One simple existing way to reward a challenge is to put a bounty on it. Although the bounty rep goes to a solution, the week during which the challenge is featured may gather extra views and votes. \$\endgroup\$ Commented May 23, 2016 at 21:15

3 Answers 3


I suggested a potential feature request related to this when we were discussing ideas about the fact that we're not a Q&A site. This is of course a suggestion that would require changes to the software, which would likely be only useful for PPCG and Puzzling.

The main idea is to introduce a separate bounty system for challenges. The main difference to the existing bounty system is that the 7-day deadline doesn't make much sense for challenges. Hence, I think that a challenge bounty system could work one (or both) of two ways:

  • Immediate assignment of a bounty to an existing challenge, in the same vein as the normal "Reward an existing answer" bounty. Sometimes you come across a challenge that you just like so much that you want to donate the author some additional reputation. It would be nice if this also resulted in the challenge being featured for some time, so that the reputation wasn't the only benefit.
  • Indefinite bounties: if you want to see a certain type of challenge that appears hard to pull off, you could offer a bounty for it. When someone posts a challenge that does this to your satisfaction, you'd award the bounty. However, I see quite a lot of problems with this. Mainly the fact that people would post their indefinite bounties, they'd never be satisfied, the user becomes inactive at some point, and then we end up with a bunch of indefinite bounties which will never be awarded and clutter the list of open bounties. Also, the fact that there's no way to ensure that a bounty is awarded for the thing it was originally offered for would become a bigger issue here.

So personally, I'd be in favour of immediate bounties for rewarding existing challenges. If someone wants to challenge others to come up with a certain type of difficult-to-do-right challenge, we could do this informally like we already do for answers.


Maintain a list of exemplary challenges

We would have a post on meta. In this post, each answer would consist of a summary of an exemplary challenge and a link. Each such answer would serve as a nomination, and challenges whose nominations that receive a certain number of upvotes would be considered exemplary.

This mechanism ensures that this honor is democratic, since any user can vote on meta, but also ensures that the honor is not unduly influenced by temporary factors, such as whether the challenge ends up on HNQ, since people who want to vote for the challenge as exemplary would have to go to meta to do it.

In addition to serving as a list of exemplary challenge, it would serve as a place for newer members of the site to look for interesting older challenges. In contrast to the Tour of PPCG proposal, it would be primarily for new challenges, to be added shortly after they are written. Of course, older exemplary challenges would still be welcome.

To increase exposure, this post could be featured on Meta, the way the Sandbox is, or mentioned in the help center or similar resources.

The reward for challenge authors would be the recognition by the community of their excellent challenge, in a place for everyone to see, and enjoy.

As an additional benefit, this system would require no changes to site infrastructure, so we could set it up ourselves.

Based on a suggestion of Sp3000: Once there are a lot of exemplary challenges, it might be hard to find them, or to find the new ones. Therefore, it would be good to have a standard template for nominations, and a stack snippet in the main post to generate the list of exemplary challenges with links, both in order of votes and in order of newness.

  • 2
    \$\begingroup\$ Are you aware of meta.codegolf.stackexchange.com/q/3626/8478? \$\endgroup\$ Commented Jun 2, 2016 at 14:17
  • 1
    \$\begingroup\$ @MartinEnder No, I think that predates me. Something like that, but maintained/promoted regularly would be great. \$\endgroup\$
    – isaacg
    Commented Jun 4, 2016 at 1:48

Trivial challenges take little time to answer, so a lot of people will answer them, which means that the challenge will frequently appear at the top of the active questions list; so, people see it, and it has a good chance of getting upvotes.
As opposed to this, a hard challenge which doesn't get any answers in the first days, "sinks" in the active challenges list; so people don't see it and it doesn't get many answers or upvotes.

The same is true for answers: answers to trivial challenges are seen more often and have a greater chance of getting upvotes than (hard) answers to hard challenges.

This is why the site gets flooded with trivial challenges, and the non-trivial ones don't get a chance to shine.

There could be a "non-trivial" tag. Or a "trivial" tag. In this way, you can filter out the trivial challenges when you want to look only at the non-trivial ones.

  • \$\begingroup\$ It is already discussed here, even if it is for a different reason :). \$\endgroup\$
    – Katenkyo
    Commented May 26, 2016 at 11:25
  • \$\begingroup\$ I think that trivial challenges are a category of challenges, like KOTH or cops'n'robbers. Trivial challenges do quickly attract a lot of answers in many languages, and the answers are of short size. Non-trivial golfing challenges attract a smaller number of answers over a longer period of time. \$\endgroup\$
    – mIllIbyte
    Commented May 26, 2016 at 11:43
  • 2
    \$\begingroup\$ What we'd like to reward is challenges that were really great, what you seems to propose is only a way to distinguish problems that were easy (and attracted lots of view/votes/answers...) from others. That would effectively put KotH in the first category as they (generally) attract their fair load of people. But keep in mind that KotHs may be the hardests challenges to write, they are the ones that ask the more time from the author, wouldn't that be insulting to categorize them with a trivial? TL;DR: the problem doesn't lie in separating "trivials" from "non-trivials" \$\endgroup\$
    – Katenkyo
    Commented May 26, 2016 at 11:54
  • \$\begingroup\$ KOTH do not attract many short answers, and are not golfing challenges, so they would not be trivial. \$\endgroup\$
    – mIllIbyte
    Commented May 26, 2016 at 12:10

You must log in to answer this question.