Sometimes, mistakes in challenges go overlooked, even after going through the sandbox. I've searched the meta, but there doesn't seem to be a meta consensus on this specific issue.

Some examples of these questions are Simple State Sales Tax Calculator and Electron Configuration.

In both cases, users posted answers following the incorrect specifications in the challenge.

By the time the challenge poster edits the question to include the correct information (but not invalidate existing answers), it may be too late, and there would be two sets of answers that follow different specifications. Plus, having these two sets of answers looks bad since some answers would technically be "incorrect".

When one encounters a challenge that contain incorrect information, what is the best course of action?

| |
  • \$\begingroup\$ Is the fault this time with elements not following Aufbau principle? But how does it not invalidate existing answers? If I understood correctly, answers already follow Aufbau principle, correct? \$\endgroup\$ – user202729 Dec 13 '17 at 7:44
  • \$\begingroup\$ @user202729 Nothing is being invalidated, as the question still includes incorrect information; the electron configurations are downright incorrect, not following the Madelung Rule. For instance 20 -> 1s2 2s2 2p6 3s2 3p6 3d2 should be 20 -> 1s2 2s2 2p6 3s2 3p6 4s2 \$\endgroup\$ – JungHwan Min Dec 13 '17 at 8:08
  • \$\begingroup\$ I see. In this case, the problem is the test cases are too large, and it takes effort to validate them. Especially when the first ~18 lines are correct... \$\endgroup\$ – user202729 Dec 13 '17 at 8:14
  • \$\begingroup\$ Related codegolf.stackexchange.com/questions/102618/… \$\endgroup\$ – Christopher Dec 13 '17 at 19:44
  • \$\begingroup\$ Same issue, wrong info for what the konami code is \$\endgroup\$ – Christopher Dec 13 '17 at 19:44

The best case, as already commented, is for the question to be closed as unclear and disputes over the specification resolved before any answers are posted. In other words, I endorse user202729's answer.

However, the question was raised in comments: what if the disputed question already has answers? Here there isn't a one-size-fits-all best approach. Depending on the individual case, the best approach might be:

  1. Leave the question in its broken state, but add a notice to the effect that the specification is known to be broken. I think this would probably be appropriate for something like the chemistry question which has one or two special cases which weren't correctly specified. The game is to golf to the spec, but it's only a game: if someone wants to take a PPCG answer and try to use it in their chemistry research then they deserve whatever bugs result from that.

  2. Leave the question in its broken state with a notice, and post a separate question with a correct spec. This is rarely going to be the best approach, but rarely is not never. If the incorrect question is an interesting challenge and the correction is significant enough that it wouldn't be a dupe, there's no harm to having two questions.

  3. Close the question, to freeze the situation and prevent more answers being added; then discuss how the change the specification. Leave comments on all of the answers warning them that the discussion is in progress, so that they can contribute. It's not nice to invalidate someone's answer without warning, but if the answerers all agree to change their answers in line with the change to the spec then the problem can be resolved while keeping everyone happy. Sometimes there may not be an option which keeps everyone happy: in that case, it's best to at least let the unhappy people feel that they were included in the process rather than presented with a fait accompli.

| |

Comment and close

Close the question as "Unclear what you're asking" to avoid answers to be posted. So answers will not be invalidated (because there isn't any).

| |
  • \$\begingroup\$ This approach sounds viable, but I think there is one issue. When an error is fixed and the post is reopened, it would most likely be buried under newer posts. Creating a new challenge is not on option because in general, we often discourage making two posts with the same content. \$\endgroup\$ – JungHwan Min Dec 13 '17 at 7:56
  • 2
    \$\begingroup\$ I think edit = pop to top of active list. \$\endgroup\$ – user202729 Dec 13 '17 at 7:57
  • \$\begingroup\$ I cannot make generalizations, but I personally prefer "newest" to "active" because "active" includes answers as well (and I am only interested in challenges). \$\endgroup\$ – JungHwan Min Dec 13 '17 at 7:59
  • \$\begingroup\$ @JungHwanMin, there is no silver bullet. Even if (to pick an extreme "solution") we went around breaking the kneecaps of the careless question setter to teach people to validate their specifications carefully, the next person who joined the site would not be forewarned of that risk. \$\endgroup\$ – Peter Taylor Dec 13 '17 at 9:10
  • \$\begingroup\$ What should happen when there are answers already? \$\endgroup\$ – JungHwan Min Dec 13 '17 at 16:36
  • 2
    \$\begingroup\$ If the OP cares about visibility of his post, then the burden is up to the OP to make sure his post is as clear as possible. If the challenge loses visibility because it was closed, that teaches the OP to look closer at their post. \$\endgroup\$ – Nathan Merrill Dec 13 '17 at 18:25
  • \$\begingroup\$ As far as existing answers: The are invalidated if they depend on the faulty spec. We try to avoid invalidation of answers, but we do allow it. \$\endgroup\$ – Nathan Merrill Dec 13 '17 at 18:26

Comment, wait, and edit (but provide evidence)

It is best to comment on the challenge and let the original poster edit the question.

The comment must include a valid evidence that shows there is indeed an error on the question and should alert other users not to post answers until the problem is resolved. For the evidence, a Wikipedia article would suffice, for example.

If there is no response after 10-20 minutes, the commenter may edit the question to include the correct information. For the edit summary, the evidence of the error should be included.

When there are existing answers, the edit should also include a note saying that existing answers are not invalidated (but no more solutions may follow the incorrect specifications).

| |
  • \$\begingroup\$ Note: the interval "10-20 minutes" is arbitrary. If you support this idea, but not this time interval, please comment suggestions. \$\endgroup\$ – JungHwan Min Dec 13 '17 at 6:01

Allow Either Specification

While the best option would indeed be to fix the challenge before any answers are posted, essentially the purpose of Sandbox in the first place, sometimes things get through. In order to avoid invalidating existing solutions while also allowing solutions that should have been correct from the start, answers should be allowed to choose either the original or corrected challenge. As an example, the State Tax challenge mentioned in the question allows answers to use either the original WS or the correct WA.

| |
  • \$\begingroup\$ @user202729 I'm... not sure what non-competing answers have to do with this? \$\endgroup\$ – Kamil Drakari Dec 16 '17 at 22:43

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .