Answering the election questionnaire got me thinking: in addition to having an objective winning criterion, all challenges also need an objective validity criterion — there's no point in comparing answers if there's a fuzzy line of what even constitutes a valid answer. This has come up several times in the discussion of popularity contests, or specifically underhanded challenges.
As a made up example of how a challenge could have an objective winning criterion but no such validity criterion, consider "Write a program that prints a funny joke. Shortest code wins." The winning criterion is standard code golf and objective, but deciding what constitutes a funny joke is very much subjective. For winning criteria, we have a close reason, but for validity criteria we don't. I was wondering whether we should decide on a close reason to use in these cases, since it's not entirely clear which one to use. (And I vaguely remember at least one case where someone closed it as "has no objective winning criterion" accompanied by a comment along the lines of "uh, well, yeah it kinda does, but it's still not objective, so...").
A few options I see:
- Unclear what you're asking: because there's no way to tell from the challenge which answers are admissible and which aren't.
- Too broad: if the definition of a valid answer is fuzzy, the challenge will allow too many answers, which won't be comparable.
- Primarily opinion-based: this is probably not what this close reason was meant for, but a challenge with a subjective validity criterion is clearly opinion-based.
- Custom off-topic reason: certainly the clearest at the moment, but we'd have to type something up each time, and I just know most people won't do it.
- Use the "objective winning criterion" reason and leave a comment, that you actually mean the validity criterion.
- Change the "objective winning criterion" reason to cover both winning and validity criteria.
- Use another close reason slot to add a separate reason for lack of objective validity criterion.
- ... something else?
I don't have a strong preference for any of these at the moment (although I might form one), so I'm looking forward to reading your opinions and arguments in the answers. :)
