I have seen things like "This is not about finding the language with the shortest approach for prime testing, this is about finding the shortest approach in every language. Therefore, no answer will be marked as accepted."
Isn't it a rule that there has to be a possibility of a best answer on Stack Exchange?
In general you're right, such questions are not an ideal fit for the Stack Exchange model. However, the exception we've chosen to make here by community consensus is that "catalog" challenges are on topic. These are intended to be a small, very specific subset of the challenges on the site, and are (as far as I know) limited to code-golf challenges only. Martin has laid out a roadmap of sorts for these here.
I'm with xnor on this one. The purpose of an objective winning criterion is that answers are comparable. Whether that is used to determine an overall winner or a winner per language isn't all that important. In particular, there is no obligation for anyone to mark an answer as accepted, so if we'd rule out challenges that only have per-language winners, I'd just "forget" to accept an answer on the challenges where I think that makes more sense.
Next, it's not that these challenges don't fit the SE model. It's that our entire community doesn't really fit the SE model. There are a lot of things that are tailored to a Q&A format, and PPCG is not a Q&A site. Likewise, there are a lot of missing features which could improve PPCG as a site for programming contests, but they make no sense for Q&A sites. In particular, if this was a proper site for programming contests, I'm pretty sure that most challenges would be scored separately for different languages just like Anarchy Golf does.
Quoting Alex
These are intended to be a small, very specific subset of the challenges on the site...
I don't know about that. I've said a few times I don't think there is even a difference between a catalogue and a code golf where the author forgets to accept an answer. It's just that the catalogues explicitly state that answers in all languages are welcome and each language should be treated as a separate challenge. To me, ideally, all challenges would be like that.
In fact, we still get people complaining about golfing languages every now and then and that they drain their motivation to answer in more verbose/"normal" languages. And every time, the experienced users tell them "you should view this as a challenge within your language (or languages of comparable golfitude), not try to beat Pyth with Java". Well, then why should catalogues be the exception? If it's such an absurd idea to compare the scores in different languages, why are we even still interested in an overall winner? Why not actually make these per-language competitions the main spirit of this community? As some of the catalogues have shown, there can be very interesting, heavily golfed answers in non-golfing languages, which normally no one bothers to look for, because they can't "win" anyway. The shortest overall language can be read off a leaderboard quite easily, so there isn't even really a loss of useful information if there is no green checkmark.
Another benefit of treating all code golf as per-language competitions is that they make that pesky "your language (version) must be older than the challenge" rule obsolete. I think that rule does more harm than good - there are rare cases where someone actually adds a built-in specifically to solve one challenge. More often than not, I find a neat solution for an old challenge in a newer language, that was in no way tailored to that challenge, my favourite being the shortest proper quine we have, in a language that was not designed with quining in mind (I believe). If every language was a separate competition, then adding built-ins to solve that challenge would be a non-issue, because the new language/version would be scored separately from all the others, and would simply make for a very boring sub-challenge. However, when that is the case, not accepting an answer is actually important, because it would likely pin a completely trivial solution to the top of the answer list.
So yes, I think that not accepting an answer should be allowed, whether stated explicitly or not, and stating it explicitly every now and then might help this community get into the spirit of making each language a separate competition, no matter how unsuitable it is for golfing.
Of course, in other types of challenges it might still make sense to compare all answers. Maybe, fastest-code has the same issues as code-golf here (in that C can always beat a Ruby answer by implementing the same approach... that's no different than porting a Python answer to Pyth). And popularity-contests don't really need an accepted answer, because that could become outdated, and you can easily find the winner by sorting by votes. But things like polyglot or rosetta-stone challenges, or metagolf where individual answers aren't even tied to a particular language, are certainly just one competition between all the answers. Accepting an answer here definitely makes sense. But that doesn't mean we should require the checkmark for all challenges just because it's there.
It's OK not to accept any answer in a code golf challenge where multiple languages participate. The important thing is that answers in the same language can be of objectively compared; cross-language comparisons do not make sense. Accepting a single answer is a feature of the SE platform that doesn't fit our purpose.
