# The state of the popularity contest tag

Recently, there's been a strong push against challenges. This post sums up the general feelings of the community pretty well, in my opinion. Even when challenge authors do everything right (like with Patch the Image), the challenges still attract close votes. The issue is beginning to divide the community, and it's time we had a definitive consensus on it.

Should challenges be closed as off-topic, and the challenge type killed (like )? Or should they remain on-topic? If so, what can challenge authors do to make their popcon challenges of good enough quality that they won't be closed as too broad?

• We just had this question two weeks ago: meta.codegolf.stackexchange.com/q/8084/8478 Or is the scope of yours different? – Martin Ender Jan 30 '16 at 20:41
• @MartinBüttner The scope is quite different. That question is about objective validity criteria. This question is about whether or not we should keep popcons as a challenge type, and if we do keep them, what we can do to make them better. – Mego Jan 30 '16 at 20:43
• I agree it's good to have this discussion separately given the belief that Objectivity gets mixed up with other aspects. – xnor Jan 30 '16 at 21:39
• Also, I think it's important that we finally do something because the current state is good nobody. – xnor Jan 30 '16 at 21:45

# Keep It, but...

...we should definitely update e.g. the tag info. The recent discussions have not been unproductive, and I think we have better guidelines than ever before but at the moment they are still all buried in meta questions/answers.

Example: As far as I know it is the first time that the term validity criterion emerged so clearly which definitely is something we need.

If you look at the recent pop-cons you will find a lot of deleted ones, most of which are just badly posed challenges even though some of them might have had an interesting core idea. But then there are still good challenges every now and then, some with fewer some with more answers but each time with interesting problems that turned out to be quite popular.

I think pop-cons should definitely stay part of this site, but we need to communicate our guidelines better.

• In addition to that, I propose we create a less ridiculous tag synonym. – mınxomaτ Feb 1 '16 at 8:52
• @mınxomaτ I don't find the tag that ridiculous. It had never occurred to me till you said it. But I see what you mean. The questions are 1. does anyone have a better idea? and 2. should we change a tag name that's been around so long? (and no I don't think popcon is better than popularity contest ;-) ) – Level River St Feb 2 '16 at 20:39
• @steveverrill 1. I like human-judged or user-judged. 2. Posterity alone is a terrible reason to do (or not to do) anything. – Mego Feb 10 '16 at 5:05
• @Mego 1. changing the name of something leads to confusion and should only be done if the reasons are truly compelling. 2. Posterity alone would be a terrible reason for something larger, such as allowing/disallowing popularity contests, but that's a different issue.. – Level River St Feb 10 '16 at 14:31

## Should the tag be split?

I think it's clear that the majority want to see popularity contests continue, but also the majority find problems with most challenges that are posted with this tag.

Is part of the problem that the tag is used for more than one thing?

There seem to be at least two different challenge types that are tagged .

There are restrictive conditions under which the challenge is to make the most popular answer. These are the most likely to be closed but also seem the best fit for the tag "popularity contest". Those that are not closed are among the most popular challenges on the site.

Then there are competitions which are not about being popular, but about achieving an objective, and popularity contest is chosen as the winning condition simply because whether and how well that objective has been achieved is down to human judgement. These seem a better fit for the site, but don't quite seem to fit the tag. Although the winner is still determined by votes, perhaps these should be called something else? Maybe someone can think of something better than , which is all I can think of at present.

### Human judged contests

Here I'm splitting based on whether there is a target. Human judged contests have a specific target (either fixed or taken as an input), and the humans are necessary in order to assess how close a solution comes to that target. Pure popularity contests have no target, just a tight restriction that makes creating a solution challenging, even though it's not technically a solution to anything.

Are there better/more ways to split vote-determined contests? Are there reasons to avoid having tags to distinguish in this way?

What I have called "pure popularity contests" are what often get closed as "just an art contest". I think having two separate tags would help to emphasise that a pure popularity contest needs to have particularly restrictive conditions to drive creativity and ensure that programming skill is demanded in order to be competitive.

• Splitting the tag has been suggested in the past. While I am in support of it, the general community consensus last time was that it wasn't a great idea. Granted, it's been over a year since then, so opinions may have changed. – Mego Feb 2 '16 at 5:22
• Interesting - I hadn't seen that one. One reason I see for having two tags is that different rules apply to the two contest types, so it would reduce confusion over what is required of each. – trichoplax Feb 2 '16 at 5:33
• While the first (and sometimes the only) kind of good popcon that comes to mind are our image processing challenges, I want to point out that both of our language design challenges have been popcons (and I personally think amazing challenges). While these are still very rare (and will likely remain rare), I'd hate to see them go in any popcon restructuring that simply forgot about them. (Alternatively, I'm all ears for objective winning criteria for language design challenges.) – Martin Ender Feb 2 '16 at 11:04
• i feel like in practice this would split the tags into "human judged questions" and "almost entirely bad questions" – undergroundmonorail Feb 2 '16 at 13:08
• @undergroundmonorail although the way they are described makes "pure popularity contests" seem too broad, in fact most of our most popular contests are of this type, and the breadth is compensated by imposing restrictions that make answering competitively difficult. I understand that some people would prefer to close all such questions though... – trichoplax Feb 2 '16 at 13:35
• @undergroundmonorail I thought that was the point. :) – Martin Ender Feb 2 '16 at 13:50
• The overwhelming success of codegolf.stackexchange.com/q/69930/8478 makes me wonder whether we even need voting-based scoring for image processing challenges any more at all. – Martin Ender Feb 3 '16 at 13:57
• @MartinBüttner That metric worked well for that specific challenge. While other challenges could also use the same objective metric for scoring, it may not be the best metric for the challenge. There will always be cases where human judging is preferable to an objective metric. – Mego Feb 4 '16 at 22:13
• @Mego I'm not saying that was a great metric. There are clearly much better metrics for image similarity. My point is rather, if even such a crude metric works this well, I'm sure we can find one that can do about as well as a human for our. purposes – Martin Ender Feb 4 '16 at 22:15
• I like the thought of a meta game progressing in the form of challenges competing to find the best metric. – trichoplax Feb 4 '16 at 22:53

# Keep it

There's nothing wrong specifically with (indeed it is a good scoring criteria for several popular kinds of challenges, such as ). The main issue is that people misunderstand that does not make your challenge exempt from the standard rules of what is on-topic here (an issue in particular is that the specification needs to define what makes something worth upvoting). by itself is a perfectly valid and objective winning criteria.

• I downvoted because there have been many apparent problems with popularity contests, both in practice and in interpreting rules, and this answer doesn't address them. I urge those who agree with "keep it" to write a more in-depth answer that explains how pop-cons should fit in, rather than just voting for this to register that you agree with its conclusion. – xnor Jan 30 '16 at 21:34
• @xnor This isn't discussion for that. This is discussion for "should we scuttle popularity contests entirely." The answer to that is no, since these are not issues that make the tag fundamentally flawed. – a spaghetto Jan 30 '16 at 21:36
• @quartata I agree with xnor. A good discussion about "should we scuttle popularity contests entirely" should make an in-depth case for how they fit in if they are to be kept. – Martin Ender Jan 30 '16 at 22:34

# Keep It

Site traffic history for ppcg.se, by month:

With a few notable exceptions (for example We're no strangers to code golf, the blip in April 2012), nearly all questions with 50k+ views have been popularity contests of one flavor or another. It is my contention that had popularity contests been a standard challenge type from early on, the site would have taken off much quicker.

I do, however, agree with other posts that a well-defined validity criterion must be in place for a question to be considered on-topic.

• +1 for stats. Also, it seems I need to start writing more popcons. My most viewed question has only 9k :( – Geobits Feb 15 '16 at 17:29
• This graph actually makes me incredibly sad. – Martin Ender Feb 15 '16 at 17:48
• That sudden spike in late 2013 was because of code trolling. Just because a question has a lot of views and/or upvotes doesn't mean that its high-quality content or even that it should be deemed on topic. – Dennis Feb 15 '16 at 17:54
• @Dennis in part, perhaps (there's exactly one post you could point to, which was completely over-shadowed by another), but it doesn't invalidate the point either way, as code-trolling was by necessity scored as popularity contest. It also gave rise to immensely popular graphical output challenges, which attract and engage a much broader audience. I think this is a good thing. – primo Feb 15 '16 at 18:21

## Retire it

The core problem that I see with is that it requires the challenge author to dictate how users should vote. This is anathema to the SE model of voting: anonymous votes which users can decide how they want to use. While there are guidelines on when to vote, they are just guidelines. One user controlling how others vote is not how SE is meant to work.

There is no compromise possible here. To embrace the SE model, users must be free to vote how they please (with the obvious exceptions of serial voting and sockpuppeting). Popcons break this rule, so they have to go. While they might work on other sites, they simply don't work on SE.

• How a voter treats an answer will depend on the question, for any SE site. No answer is judged in isolation. There are many problems with pop cons, which we need to work on, but I don't think this is one of them. – trichoplax Feb 9 '16 at 17:13
• @trichoplax The issue is not that answers are not being judged in isolation, but that popcon authors are able to direct how votes are cast. That's a bit too close to vote manipulation for my liking. – Mego Feb 9 '16 at 17:15
• I see it this way: "Can you suggest a good pancake recipe?" invites voters to upvote well written pancake suggestions, and downvote recipes for doughnuts, no matter how well written. Informing the voters on what the author is seeking is a good thing, and doesn't prevent a voter from upvoting a doughnut recipe if they wish. – trichoplax Feb 9 '16 at 17:17
• I'd be worried if there was an incentive provided to voters, but I don't think simply specifying what is being sought causes a conflict of interest. – trichoplax Feb 9 '16 at 17:19
• @trichoplax There's a considerable difference between "I need pancake recipes" and "I need pancake recipes, users should vote up gluten-free recipes even though I didn't state in the question that I needed specifically gluten-free recipes". – Mego Feb 9 '16 at 17:24
• I agree that voters should not be instructed outside the question. That seems to apply to comments though - saying "I prefer this" in the body of the question itself is just writing a question as normal. – trichoplax Feb 9 '16 at 17:47