# Things to avoid when writing challenges

If there is something that you find annoying / counterproductive / unfair / detrimental / no longer funny in challenges (question posts), describe it in an answer here, and propose a recommended response. Voting will indicate whether that response has the backing of the community.

For example, you might suggest one of the following:

1. Comment linking here
2. Edit question
3. Close question
4. Delete question
5. Other (please specify)

Consider quoting from (rather than linking to) example challenges, as the fact that you are posting here makes it likely that the challenge may be edited making the link confusing for future readers.

For aspects of solutions (answer posts) to be avoided, see instead Loopholes that are forbidden by default

• I think that this question is directly targeting the "rationales for short code lengths" posing as a broader (but mostly useless) question. I'd prefer if a different meta post were made specifically talking about the rationales, and allow the community to present both sides. – Nathan Merrill Jan 11 '16 at 14:41
• @NathanMerrill You're welcome to post a response here. Just give your reasoning and recommend no action. It should work just fine on this post. – Alex A. Jan 11 '16 at 18:32
• This may be a good thing to feature, or put in some other high traffic location so it can be treated as a reference. – J Atkin Jan 17 '16 at 1:39

# Excessively long back stories

This is a site for people who like programming challenges. They already want to see your idea for a challenge. You don't need a back story to make your challenge appeal to them. If you decide to include some context for the task, try to keep it short.

Personally, I'm more likely to be put off by a needlessly long challenge wording. If it's long because it's complex, I'll read it. If it's long because of telling a story to make a simple challenge more interesting, it actually becomes less interesting to me. Simple challenges are good. Let's keep them simple.

# Hidden pattern in the output for code golf challenges

You should not post a challenge in which an essential part is to find a pattern in the required output, which is not stated in the challenge but which greatly simplifies the code. That kind of challenges are essentially a puzzle in disguise.

As an example of a non-valid challenge, consider one that simply requires to

Produce this fixed string:

bebfjcgfdfijhjdcdiegcgeddidchjfaciiebjhbgjdjjdhfbaficajhejeefjcdahibgeagcigcaijj

without further explanation. The code will be much shorter if one realizes that the string is the first 80 decimals of pi with 0 replaced by a, 1 by b etc.

The reasons why these challenges should be avoided are:

• The actual complexity is in finding a hidden pattern, rather than in coding or programming. This is of little interest on this site, and possibly off-topic (consider posting on https://puzzling.stackexchange.com).
• Once an answer finds out the pattern, all other answers can do the same. If the pattern is going to be eventually known anyway, it is better to describe it upfront in the challenge.

So, you should state the pattern in the challenge text. If you feel that makes the challenge "less interesting", consider modifying it in some other way.

Note, however, that the challenge is not expected to describe all possible characterizations or patterns that the output satisfies. Including one (or maybe several) is enough. There is no requirement to be exhaustive.

For example, this would be valid as a challenge (adding further rules for output format etc):

Determine if the input is a triangular number, that is, if it can be expressed as 1+2+···+k, or equivalently as k*(k+1)/2, for some positive integer k.

The pattern is embodied by the definition of triangular numbers (1+2+...+k), which is included in the challenge. The challenge also contains another equivalent characterization (k*(k+1)/2). Now, an answer may find and exploit some other characterization, such as

A positive integer n is triangular if and only if 8*n+1 is a square.

It is totally fine if the challenge text does not include that information (which might have even been unknown to the challenge writer).

• Agreed: this is a programming puzzle, which isn't really a thing here anymore. Programming puzzles should do just fine on Puzzling.SE (if they're good), where "first person to solve it" is the culture. This site's culture expects that everyone, even a latecomer, has a chance to solve a challenge (more or less, though if you're not the fastest gun you may find that there are already three answers in your preferred language). – DLosc Aug 20 '20 at 3:50

# Adding special cases for the sake of completeness

This is a generalization of "complicated" number types, and is similar in scope to adding unnecessary fluff.

What I mean by this is that many challenges will try to make the problem well-defined for all possible inputs, when the challenge would have been much better if the set of inputs had been constrained more.

The challenge that prompted me to write this was multiply a string and a number. The basic idea is to repeat each character in a string a given amount of times. However, it also specifies what to do when the input number is negative, and the behavior is not a simple extension of the behavior for nonnegative inputs.

In my opinion, this special condition makes the challenge worse. I come to PPCG to solve interesting programming problems, and conditionally reversing a string based on whether a number is negative is not an interesting programming problem. It is unnecessary boilerplate that stops me from wanting to solve the challenge.

Furthermore, there are languages in which solving the problem is hard enough as it is. In the challenge linked above, one user decided to try and solve the problem in Retina. Retina was not designed for numeric handling, and the solution only worked for positive inputs. Thus, the edge condition stopped the answer from being competitive.

There are also languages such as Underload where input scope would be complete. In Underload, the most natural representation of numbers is whole numbers, and requiring that negative numbers be handled makes the challenge feel much more clunky.

In general, always choose the smallest set of inputs for which the problem is well-defined and interesting, and only require submissions to handle those. Anything else is edge cases, and handling edge cases is not fun.

• I'm with you except for your choice of example. Answers can handle both the forward and reversed cases together (1, 2 3) in clever ways for brevity,, and a challenge just to "copy each character n times" would have been nearly trivial. – xnor Jul 13 '17 at 7:57
• @xnor I think it all comes down to target language. In this case the edge-case behavior does allow for some interesting solutions in certain languages. In the context of the languages I frequently golf in, however, it's nothing but boilerplate. – Esolanging Fruit Jul 13 '17 at 8:26
• @xnor The problem, then, is that either the existing challenge or the simplified alternative would be closed as a dupe of the other, which means that there isn't a good way to balance having less but more clever solutions and having many solutions in more restricted languages. The only way I see for the second option to be preferred is if PPCG changed its voting habits and stopped upvoting more trivial challenges, something which has been brought up before but is beyond the scope of this discussion. – Esolanging Fruit Jul 13 '17 at 8:27
• @xnor TL;DR You're right. – Esolanging Fruit Jul 13 '17 at 8:37

## Assuming you've addressed sandbox feedback

When your challenge is in the sandbox and someone suggests a change or clarification, make sure your edit actually addresses their point. Too often the same complaints are brought up on the live challenge, and the poster is confused thinking they already dealt with them.

First, consider if the comment still holds true for your edited version. Maybe the rule is clearer than before, but still unclear. Maybe you've addressed some points but not others.

After editing, leave the challenge in the sandbox for more feedback. Maybe the change introduced new problems. Maybe the commenter will clarify their suggestions or offer further revisions. You can ping them in the comments to ask them to take another look.

# Exceptional edge cases

Don't require special behavior when the input is the empty list, the number 0, a negative length, etc. This makes for nasty surprises for solvers who worked on a solution only to find it's invalid on a technicality.

When possible, the edge case should follow the same rules as any other input without special instructions. However, even when the right answer is mathematically clear, it's often better to just specify the input will be non-empty, positive, etc, and avoid the issue altogether. If you do insist on including the edge case, do include a test case for it.

One place this comes up is the string representation of 0, which is conventionally '0' but mathematically is the empty string. Once in a while I find a nice recursive function that beats the boring built-in string conversion, but it fails the 0 test case, and it costs too many ugly bytes to patch. Specifying nonzero input would avoid this.

# Requiring multiple types as return value

This is very similar to Explicitly disallowing or disadvantaging arbitrary (classes of) languages though it's more subtle (and not necessarily explicit):

A lot of languages don't allow returning multiple types therefore it's not a good idea to require different types as a return value because it rules out those languages for no reason (of course there might be challenges exempt from this).

Note: While not requiring a different type, the same should hold for exit codes and errors - in some languages it is not possible to generate errors or certain exit codes.

### Examples

• Given some input X if it satisfies property Y return a list of integers, else return a falsy value
• Given some input X, if it satisfies property Y return the integers f(X), else return a string saying zzz
• Given the input X if it achieves Y return some Z else exit with a non-zero error cod
• Given X if it Ys then return some Z else error
• One suggestion to fix this is to allow option types instead. Most languages have some sort of option type. – Wheat Wizard Oct 31 '18 at 20:09
• @PostLeftGhostHunter: Yes that could solve it for a lot of "non-recreative languages" but that should be explicitly mentioned (I wouldn't call an Either Bool [Int] or Optional<Integer> truthy or falsy) and could still create problems for some esolangs. – ბიმო Oct 31 '18 at 20:14
• I've long been uncomfortable with the arbitrariness of requiring "truthy or falsy". I prefer a requirement for two distinct consistent outputs. I can't see that a more specific requirement adds anything to most challenges. – trichoplax Oct 31 '18 at 21:41
• @trichoplax: That would for example elegantly solve problems like example 1, but two distinct values is more specific than truthy/falsy. – ბიმო Oct 31 '18 at 22:13
• The problem is that sometimes there isn't an example of a falsey value for that return type. Perhaps in these cases it might be better to let users assume that the input will always be valid – Jo King Nov 1 '18 at 14:16
• @JoKing They can use (equivalent of) C++ optional type (or a pair<bool,T>) (although it depends on whether doing both input verification and computing the output at once is interesting) – user202729 Dec 10 '18 at 13:45

# Parsing expressions

In challenges dealing with algebraic expressions, don't make the golfer parse input strings like x^3-2x+1 or e^2^x, or to output this form. This is cumbersome I/O takes a lot of bytes and effort in languages that aren't like Mathematica, distracting from the actual mathematical task. For instance, a challenge about differentiating polynomials should allow them to be represented as lists of coefficients.

If you do insist on allowing a broad class of nested expressions like 2^(x+3), consider allowing them in a pre-parsed tree like ["Exp", 2, ["Plus", Var("x"), 3]]. But, make sure you give a precise, comprehensive description of exactly what operations and values can be included. It may be best to include as few operations as is necessary for the challenge to work.

Beware that not all expressions are algebraically "nice", and make sure your test cases cover the gamut of possible expressions. In particular, consider how to deal with things like x/0 or 0^0 or (-2)^(1/2) or (x*x)**0.5/x.

Make sure your problem is actually solvable. Note that there provably doesn't exist an algorithm to decide whether an expression in a real variable x is always zero for all x, if certain operations and constants are allowed. So, a golf challenge allowing a broad class of expressions might be outright impossible.

# Problems without solutions

When writing a challenge, make sure it is actually solvable within the framework of recreational programming. While a task like "Identify objects in a picture" is certainly a difficult programming challenge, it's not something people can solve in an afternoon: The entirety of the computer science field has been working on cracking that nut for decades and still hasn't found a satisfactory solution.

Therefore, when posting a challenge, you should have at least a clear idea of an algorithm that can perform the task you want, and that you could implement in a reasonable amount of time. Ideally, you'd even have written and tested it. (Such an implementation wouldn't need to be golfed, or pretty, or even fast: The points is that you can prove by example that it's possible to solve the problem.)

• One notable class of problems that falls in this category is undecidable problems, which means that they are mathematically proven to be impossible to solve with code. They are blatantly off-topic because they are literally not answerable on this site. – Bubbler May 6 '20 at 8:45
• Upgoat or Downgoat... – null Aug 20 '20 at 13:46

# Use the sandbox to "defend" your challenge

The sandbox is meant to be an aid to improve the challenge. But once posted to the main site, the quality of the challenge is entirely the poster's responsibility. Regardless of the sandbox, the challenge on the main site should be solid and well specified.

If you post to the main site and the challenge is found to have some flaw, having posted it in the sandbox first does not entitle you to anything. The fact is that the challenge has a flaw, and it should be corrected if possible, or else expect downvotes or close-votes. It's pointless to try to "defend" the challenge on the grounds that it spent several days in the sandbox and no one complained.

So why does it sometimes happen that people find the flaw on the main site and not in the sandbox? I see two main reasons:

• Fewer people see the challenge in the sandbox than on the main site;
• People who do see the challenge in the sandbox typically read it trying to find potential problems, but on the main site people try to solve it, which brings out problems more easily.

Possible solutions include:

• Try to call more attention to the challenge while in the sandbox, for example in The Nineteenth Byte chatroom.
• Ask someone to try to solve the challenge before posting.

# Exclamation marks in titles!

Quite a lot of questions have exclamation marks at the end of their titles! To me this just seems like a vain attempt to attract more attention with the hope of getting more upvotes! I don't think I've ever seen a challenge with an exclamation mark in the title where the exclamation mark enhances the challenge in any way!

• This is the only challenge I could see it being appropriate for. – user45941 Jan 13 '16 at 3:57
• @Mego Ha! yes!! – Digital Trauma Jan 13 '16 at 4:48
• The only exception I could think of are titles which happen to be quotes that include the exclamation mark in the original source. – Martin Ender Jan 13 '16 at 18:34
• youtube.com/watch?v=wyRLFWF2v_U – Robert Fraser Jan 14 '16 at 0:30
• -1 And so does any non-straightforward title. – Xwtek Jan 16 '16 at 13:52
• It means the bad challenge title is for example : "Monday Mini-Golf #6: Meeesesessess upp teeexexextext" But the good challenge title is : "Do the webtollkit messing tranform in shortest code possible." – Xwtek Jan 16 '16 at 14:01
• @Christian actually, no. I'm only attempting to address exclamation marks here as IMO they are in their own category of pointless. I am not trying to address other styles of title - interesting, creative, boring or otherwise – Digital Trauma Jan 16 '16 at 17:53
• If the title is an imperative sentence (Make a slow error quine maker!) or an exclamation (My god, it's full of spaces!), I fail to see the problem. – Dennis Jan 19 '16 at 17:15
• @Dennis To me, imperative sentence with exclamation mark implies forceful command. Perhaps we're different at PPCG, but in my experience forceful commands don't go over that well in questions on other .SE sites. Fair point about your second example though. – Digital Trauma Jan 19 '16 at 17:26
• The only time I've done this are to get around the 15 char limit for titles and I don't see how it really causes any harm to the actual challenge. – Downgoat Jan 20 '16 at 2:52

# Consider if your challenge really needs random output

Challenges that require output require extra clarifications that challenges with deterministic output do not, and additionally exclude languages with no source of non-determinism.

Often times randomness only acts as a cumbersome output format, adding the randomness boilerplate from whatever language (or preventing submissions in that language).

At the same time there are few benefits that it provides to a challenge.

If there are a finite number of valid outputs for every input (which seems to be the majority of challenges) you can change a challenge from

Generate a random X

to

Output all Xs

Of course there are other ways to remove unnecessary randomness, for example if challenges of the form:

Generated a random X that satisfies property Y

often (but not always) break down into two challenges

Generate a random X

and

Given an X determine if it satisfies property Y

And you will often find a more worthwhile challenge in one of these two (or both).

Of course there are challenges that are best with random outputs. The clearest case of this are challenges that ask to generate output with a specific distribution, where meeting the specific distribution is the challenge.

Some examples of such challenges

# Requiring time limits

This may differ depending on your opinion, as quite a lot of challenges have time restrictions, but, for me, this is a form of requiring a minimum score.

If a challenge is , unless the specific goal of the challenge is to do the task quickly, adding a time limit simply prevents people from posting good answers and adds nothing positive to the challenge. Older challenges are more guilty of this, but occasionally a new challenge will be posted requiring answers to finish within X minutes.

A lot of challenges that contain a time limit often do so in an attempt to make the challenge more interesting. Banning specific builtins, classes of languages or other arbitrary things has been repeatedly shown to not make a challenge more interesting (see the other answers in this thread), so requiring answers to meet a specific time limit is unlikely to as well.

Furthermore, timing is non-perfect. What takes 5 seconds on one computer may take 50 on another. That's why we require specifications to be provided for , otherwise it's no longer objective. TryItOnline has a time limit of one minute, so could possibly be used for such a time restriction. However Dennis - the host of TIO - has said repeatedly that TIO isn't reliable enough for , and so probably isn't accurate enough for time limits.

The worst culprits in my experience are chameleon challenges, which detail an inefficient algorithm/method to complete the task, then all but outlaw that method by requiring answers to meet a time limit (see a lot of the older challenges for example)

Obviously, this is a matter of opinion; some people prefer to see solutions finish, others don't mind. However, when considering adding in a time limit, ask yourself if it actually improves the challenge or just prevents otherwise valid answers from being posted.

As an aside, this isn’t to say that you should never include a time limit. Challenges where the sole aim is to “Do this thing quickly and golfy” are the main areas where a time limit works. Challenges where the aim is “Do this thing golfy. Oh and make sure you choose an algorithm that isn’t too slow” are a poor idea. Instead, look into either , or consider using an asymptotic complexity limit, , which is much more objective.

• Instead of time limits, is there a better way to say “Don’t just brute force the answer”? Or do you think people shouldn’t even try to stop brute force solutions? – water_ghosts Dec 18 '20 at 21:57
• @water_ghosts “no brute forcing” is, IMO, an unobservable requirement (another thing to avoid). I don’t think people shouldn’t try to stop brute forcing (and similar), I just think that a lot of good challenges are ruined by adding an unnecessary time limit. If a challenge works well with a time limit, good for it, but that doesn’t mean most don’t – caird coinheringaahing Dec 18 '20 at 22:11
• Can't we just assume most answers will be in good faith? Those that do (purposely) use brute force just won't be upvoted as much. – user Dec 18 '20 at 22:35
• @water_ghosts Asymptotic time complexity is a observable requirement even if it is not computable. So if you need to ban brute force solutions you can simply require that all solutions be of a certain big O complexity class. However I would seriously consider why you want to ban brute force answers and make sure that it would make the challenge more interesting. – Wheat Wizard Dec 19 '20 at 12:13
• I think time limits have a good use in combinatorics golf challenges where they prevent boring brute-force try-everything solutions and so encourage golfing that is actually specific to the problem. For these challenges, limiting computational complexity is usually overkill (and can be tricky to specify rigorously), and the difference between code that takes 2^1000 operations and 1000 operations usually makes specifics of timing not matter. – xnor Dec 20 '20 at 0:11

# Multipart challenges with unrelated sub-tasks

In general, a challenge on Code Golf should focus on a single core task. Challenges with multiple sub-tasks are discouraged, especially when the sub-tasks have zero or very little interaction or common aspects.

If the answer to at least one of the following questions is "yes", your challenge has a high chance to be poorly received on Code Golf.

1. If the sub-tasks are posted as individual challenges, are they interesting on their own, or are some of them already on Code Golf?
• If multiple individual sub-tasks are already on Code Golf, your challenge has a high risk of getting closed as duplicate. An extreme example.
2. If you collect golfed answers to individual sub-tasks and simply merge into one, is it likely to be competitive?

One type of newbies' challenges is "replicate this small but multi-function utility", which suffers from being multipart by definition. (Note that they often also suffer from other kinds of quality problems, such as rigid I/O, input validation/error handling, and sometimes lack of clarity due to random numbers or arithmetic expressions.)

More in-depth discussion can be found in this post by Martin Ender.

## Pure programming puzzles

Despite our name, programming puzzles aren't generally welcome here. I'm talking about challenges where you're given a task with some impossible-seeming restrictions and must use some clever trick or language feature to solve it.

Programming puzzles are closed most of ther time for not having an objective winning criterion: something that lets you compare submissions to decide a winner. A criterion for a valid solution is not enough -- others need to be able to do better and beat the solution. You could say the first valid solution wins, but some people find it unsatisfying that someone else can win before they even see the challenge. It's better when more people can participate in and enjoy a challenge.

If it makes sense for your challenge, you can say the shortest solution wins (code golf) or have some other metric. This is best for puzzles that can be solved by more than one method, so the search for better methods can be interesting. Beware that once someone posts a solution to the puzzle, others can see it and copy the key idea with a better-scoring implementation.

• As a sidenote, I don't think programming puzzles should necessarily be off-topic. It seems that whether pure programming puzzles are on-topic is still up for debate, but the reality seems to be that such questions get closed, so it's kinder to well-intentioned newbies to warn against posting them. – xnor Sep 13 '16 at 7:30
• For those not familiar with the history: Is the name of the community correct? showed strong community support that "Puzzles" in the site name should be changed, so we would be "Programming Contests & Code Golf", which would allow all types of competitive contests, and avoid puzzles without a winning criterion. – trichoplax Sep 13 '16 at 12:59
• The more recent discussion Should we change our name showed more support for keeping the current name, but there are lots of different issues mixed up in choosing a name, so I'm glad this answer now provides a place for voting on whether programming puzzles without a winning criterion should be avoided. – trichoplax Sep 13 '16 at 13:03
• To reduce the problem of instant voting without reading the full content, would it be useful to change the title to "Programming puzzles without a winning criterion" or "Programming puzzles without a scoring method"? – trichoplax Sep 13 '16 at 13:05
• @trichoplax Good idea, I changed it to "Pure programming puzzles". – xnor Sep 13 '16 at 17:03
• From codegolf.stackexchange.com/help/on-topic : "Non-challenge questions that are related to solving programming puzzles or a particular type of challenge are also on topic." – gaazkam Oct 29 '17 at 0:01

## Entirely nonsensical rationales for short code length.

This was fun (and was documented as a local meme) when the rationales vaguely made sense, for example, "We want the program that does this to be as small as possible so it can fit into the margins" (from The Margin is too Narrow).

However, this has gotten out of hand. Recently more challenges have included rationales that make absolutely no sense, in or out of context. For example, "Because you can only stack about 5 goats before they start to fall, your code will need to be as short as possible" (from Tokenize a Stack-Based language, since edited out).

In an effort to keep challenge specs concise and readable, I propose that we do the following when we see something clearly off-topic for the challenge in the spec:

• Edit the question
• Link here in the edit description (it will appear in the revision history)
• Send attack drones
• "Send attack drones" is an entirely nonsensical solution to these entirely nonsensical rationales. :P – ETHproductions Jan 10 '16 at 21:48
• Just wondering, did Trichoplax post this question just so you could post this answer? Are you Trichoplax in disguise? – Level River St Jan 11 '16 at 1:57
• Chemistry-tagged challenge: Because there are only 6.022140857×10^23 atoms in a mole, your co... (yawn) :-D – Level River St Jan 11 '16 at 2:00
• @steveverrill Haha no, I think trichoplax would have posted this question anyway, though I think if I hadn't posted this answer, he would have. ;) – Alex A. Jan 11 '16 at 2:07
• What counts as entirely nonsensical? I've used "However, I've gotten lazy in my old age. So I'm outsourcing it to you, and disguising it as a code golf challenge.", which I think could count as either nonsensical or perfectly reasonable, depending on whether you believe it or not. – Geobits Jan 11 '16 at 3:58
• @Geobits That's not a rationale for short code length though. But I think people should use judgment as to whether something is entirely irrelevant to the spec and should be removed. – Alex A. Jan 11 '16 at 4:21
• The only problem with individual judgement in this case is that something only has to annoy one person to result in an edit, even if no one else finds it annoying. There's something to be said for @NathanMerrill's suggestion in the comment on this meta question. – trichoplax Jan 11 '16 at 15:00
• I suggest downvoting. This would be too subjective if people begin to look for workarounds. Having a dumb rule may make it seemed interesting looking for workarounds to some askers. – jimmy23013 Jan 11 '16 at 15:06
• i don't care enough to vote but i will say that i'd feel kind of weird about a rule dictating what kind of jokes you're allowed to tell on this site – undergroundmonorail Jan 11 '16 at 15:33
• I don't think this answer is a perfect fit for the question, because "no longer funny" implies that it was funny once... – Peter Taylor Jan 11 '16 at 16:09
• @PeterTaylor this is a good point, and makes me realise I phrased the question poorly. I've edited to hopefully improve this. – trichoplax Jan 11 '16 at 18:20
• What about the ones that make sense? Also most of them are at the bottom of the post, I'm not sure how that affects readability. – J Atkin Jan 15 '16 at 3:10
• Wait, are you saying that just because you don't find the ridicuous jokes funny but I do, I'm not allowed to put them in my challenge rationale? pah – cat Jan 18 '16 at 7:40
• @cat It's not because they aren't funny, it's because they clutter the specs. Short code rationales add nothing to challenges. – user45941 Jan 26 '16 at 22:55
• They add humor (subjectively), in much the same way that things like struck-through "instructions" for the puppy command do, for instance. I'd be hard pressed to say the puppy commands add less clutter, given that they're within the actual instructions and not as a fore/afterword. – Geobits Mar 24 '16 at 16:35