As I see it, there are five types of invalid answers:
Answers that produce incorrect results.
This is the most common type, and usually an accident.
Answers that produce correct results, but break a rule of the challenge, ignore parts of the spec or violate a loophole.
For example, answers that make use of banned built-ins fall into this category.
Answers that produce correct results, comply with the rules of the challenge, but aren't a serious contender for the winning criteria.
For code golf, e.g., this is limited to answers that do not even attempt to golf the code. Answers that are simply poorly golfed are not invalid.
Answers that have one of the following, fatal flaws:
- They exist only to promote a product or service and do not disclose the author's affiliation.
- A reasonable person would find their content inappropriate for respectful discourse.
Answers that aren't actually answers.
This includes anything but code that is intended to solve the challenge.
Marking an answer as non-competing does not exempt it from being invalid.
Answers of type 4 and 5 aren't actually answers. The policy across the entire network is that they should be deleted on sight, and there's no reason to make an exception here.
I think all other types should be removed as well, with varying levels of urgency. Even without an official policy, this is commonplace across the entire network. While these answers should not be flagged on other SE sites, high rep users delete them all the time.
If the answer is invalid without the poster's knowledge, it shouldn't be deleted without notifying the poster first and giving him some time to fix his mistake.
If the answer is deliberately invalid (most common for type 2), there really is no reason to wait or tell the poster something he already knows.
If you figured out or have been told that your answer is invalid, I think self-deletion is the best option.
Deleting the answer quickly will prevent it from accumulating downvotes, which may or may not get removed after the answer has been fixed.
Deleting the answer yourself means that you can undelete it just as easily.
In contrast, undeletion will require moderator intervention if the answer has been deleted by others.
While deleting your answer will invalidate any rep your may have earned from it, you'll get it back if you fix the answer.
Not an answer flags were creates specifically for type 5 answers, and should be cast when they are encountered.
Likewise, spam and rude or abusive flags were created for type 4 answers, and should be used instead of not an answer flags.
Very low quality flags pretty much apply to all other types, but can only be cast on answers with a non-positive score. They are also cleared automatically when the post is edited, so they may be ineffective if the intention is to get a post removed.
This leaves flagging as in need of moderator intervention.
The policy across the entire network is that flags should not be used to indicate [...] an altogether wrong answer, but I think our site should be (once again) an exception.
First of all, we have a huge moderator-to-activity ratio. Stack Overflow moderators cannot be expected to complete the time-consuming task of deciding whether an answer is valid or not, but our flag volume is rather low and checking PPCG answers is usually easier. A flag that clearly explains why the answer is invalid (i.e, which rule it violates or which test case it fails) does not require an unreasonable amount of moderator time.
Also, unlike other sites, I don't think we need examples of what not to do. There is absolutely no merit in keeping an answer that violates a rule of the challenge; they don't teach anything. If an answer passes all test cases but still produces incorrect results, there is no benefit in keeping the answer after a new test case has been added to the question.
But it is important to wait until the moderator can actually take action. I think deleting an accidentally invalid answer on sight is an unnecessarily unpleasant experience for the poster, and undeleting it requires yet another moderator intervention.
Voting to delete
Trusted users (currently 20k+ rep) can cast deletion votes on answers. In all of the above situations that suggest flagging, trusted users can also vote to delete an answer.
With three votes from trusted users, involving a moderator becomes unnecessary. However, trusted users should flag instead or on top of their delete votes unless all of the following conditions are met:
The answers has a negative score.
This is a hard requirement, since even trusted users cannot vote to delete answers with non-negative scores.
The challenge is still active.
Trying to gather three delete votes for an answer on a question that barely gets views anymore will take a long time. The moderation tools (10k+ rep) could help with this, but barely anybody seems to use them.
The answer should be removed entirely, not converted into a comment.
Only moderators can convert answers into comments.
The answer is neither spam nor rude or abusive.
Six "red" flags will not only delete the post, but hide it in the revision history and apply a 100 rep penalty to the poster. Deleting the post via deletion votes would prevent this.
Immediately flag invalid answers of type 4 as spam or rude or abusive.
If your own answer turns out to be invalid, delete it yourself.
Immediately leave a comment on all invalid answers that are not your own.
Immediately flag invalid answers of type 5 as not an answer.
If you are a trusted user, the challenge is fairly active and the answer should not get converted into a comment, consider voting to delete instead.
Give the poster time to fix or self-delete his invalid answer of type 1 to 3.
If a sufficient amount of time has passed (say 48 hours) or the answer was deliberately invalid, flag the answer as in need of moderator intervention, providing an explanation of why it is invalid.
If you are a trusted user and the challenge is fairly active, consider voting to delete instead.
If you fix your self-deleted answer, undelete it.
If you fix your answer and cannot undelete it yourself, flag it as in need of moderator intervention, requesting its undeletion.