# What should we do with challenges with incorrect information?

Sometimes, mistakes in challenges go overlooked, even after going through the sandbox. I've searched the meta, but there doesn't seem to be a meta consensus on this specific issue.

Some examples of these questions are Simple State Sales Tax Calculator and Electron Configuration.

In both cases, users posted answers following the incorrect specifications in the challenge.

By the time the challenge poster edits the question to include the correct information (but not invalidate existing answers), it may be too late, and there would be two sets of answers that follow different specifications. Plus, having these two sets of answers looks bad since some answers would technically be "incorrect".

When one encounters a challenge that contain incorrect information, what is the best course of action?

• Is the fault this time with elements not following Aufbau principle? But how does it not invalidate existing answers? If I understood correctly, answers already follow Aufbau principle, correct? Dec 13, 2017 at 7:44
• @user202729 Nothing is being invalidated, as the question still includes incorrect information; the electron configurations are downright incorrect, not following the Madelung Rule. For instance 20 -> 1s2 2s2 2p6 3s2 3p6 3d2 should be 20 -> 1s2 2s2 2p6 3s2 3p6 4s2 Dec 13, 2017 at 8:08
• I see. In this case, the problem is the test cases are too large, and it takes effort to validate them. Especially when the first ~18 lines are correct... Dec 13, 2017 at 8:14
• Dec 13, 2017 at 19:44
• Same issue, wrong info for what the konami code is Dec 13, 2017 at 19:44

## 4 Answers

The best case, as already commented, is for the question to be closed as unclear and disputes over the specification resolved before any answers are posted. In other words, I endorse user202729's answer.

However, the question was raised in comments: what if the disputed question already has answers? Here there isn't a one-size-fits-all best approach. Depending on the individual case, the best approach might be:

1. Leave the question in its broken state, but add a notice to the effect that the specification is known to be broken. I think this would probably be appropriate for something like the chemistry question which has one or two special cases which weren't correctly specified. The game is to golf to the spec, but it's only a game: if someone wants to take a PPCG answer and try to use it in their chemistry research then they deserve whatever bugs result from that.

2. Leave the question in its broken state with a notice, and post a separate question with a correct spec. This is rarely going to be the best approach, but rarely is not never. If the incorrect question is an interesting challenge and the correction is significant enough that it wouldn't be a dupe, there's no harm to having two questions.

3. Close the question, to freeze the situation and prevent more answers being added; then discuss how the change the specification. Leave comments on all of the answers warning them that the discussion is in progress, so that they can contribute. It's not nice to invalidate someone's answer without warning, but if the answerers all agree to change their answers in line with the change to the spec then the problem can be resolved while keeping everyone happy. Sometimes there may not be an option which keeps everyone happy: in that case, it's best to at least let the unhappy people feel that they were included in the process rather than presented with a fait accompli.

# Comment and close

Close the question as "Unclear what you're asking" to avoid answers to be posted. So answers will not be invalidated (because there isn't any).

• This approach sounds viable, but I think there is one issue. When an error is fixed and the post is reopened, it would most likely be buried under newer posts. Creating a new challenge is not on option because in general, we often discourage making two posts with the same content. Dec 13, 2017 at 7:56
• I think edit = pop to top of active list. Dec 13, 2017 at 7:57
• I cannot make generalizations, but I personally prefer "newest" to "active" because "active" includes answers as well (and I am only interested in challenges). Dec 13, 2017 at 7:59
• @JungHwanMin, there is no silver bullet. Even if (to pick an extreme "solution") we went around breaking the kneecaps of the careless question setter to teach people to validate their specifications carefully, the next person who joined the site would not be forewarned of that risk. Dec 13, 2017 at 9:10
• What should happen when there are answers already? Dec 13, 2017 at 16:36
• If the OP cares about visibility of his post, then the burden is up to the OP to make sure his post is as clear as possible. If the challenge loses visibility because it was closed, that teaches the OP to look closer at their post. Dec 13, 2017 at 18:25
• As far as existing answers: The are invalidated if they depend on the faulty spec. We try to avoid invalidation of answers, but we do allow it. Dec 13, 2017 at 18:26

# Comment, wait, and edit (but provide evidence)

It is best to comment on the challenge and let the original poster edit the question.

The comment must include a valid evidence that shows there is indeed an error on the question and should alert other users not to post answers until the problem is resolved. For the evidence, a Wikipedia article would suffice, for example.

If there is no response after 10-20 minutes, the commenter may edit the question to include the correct information. For the edit summary, the evidence of the error should be included.

When there are existing answers, the edit should also include a note saying that existing answers are not invalidated (but no more solutions may follow the incorrect specifications).

• Note: the interval "10-20 minutes" is arbitrary. If you support this idea, but not this time interval, please comment suggestions. Dec 13, 2017 at 6:01

# Allow Either Specification

While the best option would indeed be to fix the challenge before any answers are posted, essentially the purpose of Sandbox in the first place, sometimes things get through. In order to avoid invalidating existing solutions while also allowing solutions that should have been correct from the start, answers should be allowed to choose either the original or corrected challenge. As an example, the State Tax challenge mentioned in the question allows answers to use either the original WS or the correct WA.

• @user202729 I'm... not sure what non-competing answers have to do with this? Dec 16, 2017 at 22:43