7
\$\begingroup\$

It seems as if my questions either acquire very few answers, and only from the most advanced coders on this site, or many answers from every coder on the site. This implies that my challenges are either very difficult or very easy.

How can I craft my challenges so that they are not too difficult and not too easy? In other words, how do I target middle-level coders?

\$\endgroup\$
7
  • \$\begingroup\$ Just make them easier (but not boring). That should do ;-) \$\endgroup\$ Apr 1, 2014 at 6:27
  • 2
    \$\begingroup\$ @JanDvorak The point is: how? I'm terrible at this because the interesting parts of the challenges I come up with are usually the difficult parts, so cutting them out would be removing the reason why I made the challenge. \$\endgroup\$
    – Justin
    Apr 1, 2014 at 6:51
  • 2
    \$\begingroup\$ Can you enumerate some questions which you think hit the sweet spot? \$\endgroup\$ Apr 1, 2014 at 15:36
  • 3
    \$\begingroup\$ With the hottness ordering of the default front-page view and the "Hot network questions" sidebar, there is a fair amount of positive feedback for attention to questions built into the SE model. It is possible that the bimodal distribution you see in number of answers partially explained by that feature of the network. \$\endgroup\$ Apr 1, 2014 at 15:36
  • \$\begingroup\$ In my opinion it's not really surprising that the hard parts are the most interesting ones. A lot of people (including me) like to invent new approaches rather than applying concepts they already know. \$\endgroup\$
    – Niklas B.
    Apr 7, 2014 at 17:45
  • \$\begingroup\$ I think that codegolf.stackexchange.com/q/25314/9498 is somewhere in the middle of difficulty. \$\endgroup\$
    – Justin
    Apr 7, 2014 at 18:27
  • \$\begingroup\$ Maybe we could deal with this in code golf the same way as in real golf - with handicaps. (and yes in real golf you have to have played a few games to get a handicap, the 'pretend to be a newbie' problem exists there too) \$\endgroup\$
    – bazzargh
    Apr 14, 2014 at 23:32

1 Answer 1

6
\$\begingroup\$

Middle level may not be a realistic target

It is difficult to aim for a specific mid range ability level. The problem is that someone who is middle level in one aspect may be unfamiliar with another aspect, and an experienced expert in yet another. While there are some admirable people who happen to be good at everything, most people have strengths in certain areas and weaknesses in others. This means that if you ask what you consider to be a middle level question, it may turn out to be far too easy or far too difficult for the majority of users.

Instead I would recommend trying to create questions which appeal to a broad range of abilities. Perhaps a question where a technically correct answer is trivial to produce, but an answer which can compete with the best requires far more.

The question you linked to in a comment under your question has a bonus if the running time is independent of n. Having this as a bonus rather than a requirement allows people to join in even if they can't meet the condition (you can see that some have in this case). This encourages more answers while still keeping it challenging for those who want to seek the bonus. In particular, I imagine that having some lower level answers included will encourage new users or browsers to start submitting answers of their own, even if they are not yet confident.

I like competitions for this reason - at the start even the least knowledgeable of users can submit a simple answer which will do well in the early environment. Then other users can aim to beat the simple early answers, which some people find less off-putting than trying to come up with an optimal solution immediately. For example, questions like n-d tic tac toe and create your wolf.

Gradually growing solutions

Also bear in mind that someone who has the skill and imagination required to create the best solution to your question may not submit an answer at all unless they are first drawn in by the idea of a "quick solution". Just because they are capable of an elaborate solution doesn't necessarily mean they will want to spend that much time on your question when they first see it and it is a stranger to them. If you design your question to allow quick rough solutions then such imaginative people may be tempted to throw a solution together in 10 minutes, which will get them interested and familiar with the problem. From there they might decide to improve their solution as others are submitted that beat it. Eventually they may submit the full spectacular solution that they might never have written if not for the initial rough attempts.

This effect can be strengthened by providing an easy to beat answer as an example. This can be done in a contest by providing an example solution which anyone can beat. In a challenge you could post example source code. As well as making absolutely clear what is required, seeing the source code is also likely to spur people on to beat it. In both types of competition, once the answers that beat the example start coming in, the difficulty will start to increase leading to the really interesting answers.

Edit: Following the good advice in Peter Taylor's comment below, I no longer recommend including the example in the question. Simply submit it as an answer. If you refer to it in the question you may want to include a link, as the example answer isn't likely to stay at the top for very long...

Asking good questions is a real challenge

Some questions will have one easy answer and no potential to improve on it. Others will have a huge variety of fascinating different potential approaches. It's hard to give a guide to how to create questions that allow for varied approaches because it requires the same creativity to distinguish between such questions as it does to answer them. That is, sometimes the only way to find out if a variety of approaches is possible is to show the question to the community. Seeing the potential for a variety of different approaches as just one person is more difficult...

This is why asking a good question on Programming Puzzles and Code Golf is much harder than asking a good question on other Stack Exchange sites. When you find it difficult, that is because it is difficult. Don't let this discourage you - a constant supply of good questions is what makes this site work. If you think of a question and you're not sure if it will attract good quality answers, submit it and find out - either as a question or search for sandbox in meta if you want guidance first. I believe that continually submitting and getting feedback is the way to find good questions, as there are no magical shortcuts.

Setting programmers against each other will generally be more interesting than setting them against a fixed problem, as it leaves things open ended. However, short well defined questions that can be answered in 10 minutes are also important, as not everyone has a hour to commit every time they drop in to the site. A good supply of long AND short problems is essential.

\$\endgroup\$
4
  • 2
    \$\begingroup\$ Excellent answer. There's just one small point I disagree with: rather than cluttering up the question with a reference answer, I think it's better to post it as an answer (maybe community-wikied). \$\endgroup\$ Apr 14, 2014 at 8:39
  • \$\begingroup\$ Thank you @Peter Taylor that sounds more clear and elegant - I've edited the answer to reflect this. \$\endgroup\$
    – trichoplax
    Apr 14, 2014 at 12:46
  • \$\begingroup\$ "Setting programmers against each other will generally be more interesting than setting them against a fixed problem," Hmmm...yes. Perhaps that is contributing to the slowly growing popularity of [king-of-the-hill]. Though the complexity of writing and managing the tasks obviously works against them. \$\endgroup\$ Apr 14, 2014 at 23:43
  • \$\begingroup\$ @dmckee yes they don't tend to be quick challenges (there are plenty of quick answers but they don't do so well...). I don't see this as a disadvantage though - I think having a mixture of quick and slow challenges gives an individual something for different types of free time. A quick problem in a lunch break or something to get their teeth into over a weekend. I think having both will build a longer relationship with the site. \$\endgroup\$
    – trichoplax
    Apr 15, 2014 at 7:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .