When should rule changes count?

Question

Related: When is it allowable to change the rules?

As noted in the linked meta post above, it's typically particularly frowned upon to significantly change the rules of a challenge after posting it. There's no hard and fast rule on this yet, so the purpose of this meta post is to enact at least some kind of policy to determine when drastic edits should be rolled back.

Here, we define "rule change" as not any type of edit but any edit that causes one or more reasonable hypothetical answers to become invalid beyond repair. Note that the "beyond repair" clause means that this does not include minor rule changes such as input/output format. Note that the wording of this definition also (intentionally) does not include edits that give answers additional liberties.

There are a few "poll-style" answers here to vote / comment on:

• Should rule changes that don't invalidate existing answers be allowed?

• Should rule changes that invalidate one or more existing answers be allowed?

• Do rule changes in comments count?

Adding your own answer is encouraged, though, possibly addressing some of the following questions:

• Exactly what it says in the title: If you do think that some rule changes should be acceptable, which ones specifically and in what situations?

• If we're going with "rule changes are only allowed if they're minor enough," how do we define a "major" rule change?

• If you think the policy should be something other than something that was mentioned here, what is it, and who gets to decide which rule changes are allowed?

Answer 1

Rule changes are allowed

They're often a very bad idea, but I don't think we should forbid them, regardless of whether they invalidate answers or not.

Most of the time, rule changes are meant to improve the challenge. This can include closing loopholes as isaacg mentioned, but it can also mean just tightening or relaxing the rules slightly if they are found to allow uninteresting solutions which make the interesting ones uncompetitive, or if they are found to disallow interesting solutions.

The challenge that sparked this discussion was this one. That's a very good example of a rule change that improves the challenge. In it's original form, brute force was allowed, which would obviously never complete for a square of any interesting size. But efficient solutions exist, and they are a lot more interesting. So adding a time limit after answers were already posted (especially ones which just created random grids until one was valid), definitely improved the question. Early uninteresting answers should not have the power to "lock" a bad challenge in its current state.

Yes, invalidated answers are frustrating, and everyone whose answer gets invalidated by a rule change is well within their rights to downvote the challenge (although I personally find it very weird that PPCG is the only SE where you risk being downvoted for improving your post), but if that allows the challenge to be a better challenge, then I think it's worth the change. In any case, there shouldn't be a blanket rule against changes, because I think it will lead to more abuse (by early answerers) than it will do good.

Challenge authors should still think twice (or three times) if a rule change is worth the trouble. If it invalidates a lot (or the majority) of answers, it's probably not worth it, because many of those invalid answers will remain and it will become a pain to pick out the valid ones. In that case, just learn your lesson for next time. But if it's about disallowing some boring approach that essentially breaks the challenge and has been used once or twice, then go ahead.

As for answerers whose answers are invalidated, be a good sportsman and delete (or change) your answer. And before you downvote the challenge for the rule change, consider whether your solution actually added anything valuable to it or whether the challenge isn't actually more interesting if answers like yours aren't valid any more.

Answer 2

In support of allowing rule changes that invalidate existing answers

Closing loopholes

Suppose you've posted a fastest-code challenge about finding factors of integers. Suppose the description of the challenge is "You will be given a positive integer. You must output a proper factor of this integer as quickly as possible."

Now, suppose that someone answers with a submission that always returns 1. Technically, this is a valid solution to the challenge as stated. However, it is a trivial solution, and nothing of value is lost by changing the question to say "proper factor greater than 1". This does invalidate the existing solution, and it is a rule change, but it brings the rules in line with the clear intent of the poster and improves the challenge with no real loss.

Obviously, there is a fine line between closing loopholes and real changes to the rules, but we should allow closing loopholes, even if answers which exploit them have been posted.

Answer 3

Rule changes that don't invalidate existing answers should be allowed (they should not be rolled back).

(Upvote if you agree, downvote if you disagree, and comment for anything else.)

Answer 4

Rule changes that invalidate existing answers should be allowed (they should not be rolled back).

(Upvote if you agree, downvote if you disagree, and comment for anything else.)

Answer 5

Rule changes in comments are equally valid as those that have been edited into the actual post.

(Upvote if you agree, downvote if you disagree, and comment for anything else.)