13
\$\begingroup\$

2 years ago, we discussed dropping the objective winning criterion requirement. No consensus was really reached. I'd really like to start that conversation again, as I feel like it is hindering our growth into a more comprehensive site.

What advantages/disadvantages are there to having objective winning criteria?

I'll list the ones that I personally see:

Disadvantages

The primary problem I see is that it hinders challenges that are looking for efficient solutions or (for particularly tough challenges) any solution at all.

Most of the time, we get around the problem by simply making the challenge , and adding time restrictions. Ideally, the problems would be , but because we have this "objective winning criterion", the user needs to commit to running the code themselves.

Advantages

It helps us identify and close off-topic questions. We still need answers that work towards a single goal.

Side note: I'm not looking to poll the community as to whether we should actually remove it, but I am looking for a discussion about it.

\$\endgroup\$
  • 1
    \$\begingroup\$ Shouldn't your own answer to the question you ask go into an actual answer? \$\endgroup\$ – Martin Ender Jan 17 '16 at 16:57
  • \$\begingroup\$ I'm looking to start a conversation, not "vote on whether we should do it". I was using my list of disadvantages/advantages as a starting point, so it makes sense to me that I should keep them in the question. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 17:09
  • 2
    \$\begingroup\$ I'm not sure this has anything to do with voting (except that currently votes on the question are more likely votes on your opinion on not on whether having this discussion is a good idea). You've got an opinion on your question, and that seems like it would fit in an answer. I don't see how it could be detrimental to starting a discussion if the same text is found in an answer. \$\endgroup\$ – Martin Ender Jan 17 '16 at 17:17
  • \$\begingroup\$ @MartinBüttner I agree it would certainly fit in an answer. It simply makes more sense to me that the starting point of a conversation goes in the question. I'm willing to move it though. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 17:19
29
\$\begingroup\$

Dennis's answer touches upon this, but I wanted to write a separate answer to make it more explicit:

Objective scoring is essential to drive competition, and competition inspires creativity.

Here is how a challenge works that doesn't have objective scoring: you come up with a solution that is valid, and may or may not do reasonably well. You think about how you can improve its quality (whatever "quality" means in that case), and if you can, you do. At some point you'll think "yeah that looks reasonably good, I'll go with this". And that's the end of it.

Here is how a challenge works that has an objective scoring: you come up with a solution that is valid, and may or may not do reasonably well. However, you have a pretty good idea how you can or cannot improve that score, and if you can, you do. At some point you'll think "yeah that looks reasonably good, I'll go with this". But then you'll see the other participants' scores and notice that you're only a few "points" behind (whatever those are in the specific challenge). And suddenly you've got a very specific goal: come up with something to beat the other score. That's a tangible goal and a motivation to try new things, and it eliminates the problem of thinking "yeah I think this is as good as it can be" before that's really the case.

And the interesting thing that the score alone is enough to create this motivation - there doesn't even need to be competition. As soon as you've got a score, you've got your own score to beat. This applies a) when your solution is the only solution to the challenge (so far), b) your solution has such a bad score that it's nowhere near competitive (e.g. because you're participating in a code golf with C#), or c) your solution is by far the best solution already and none of the other submissions currently provide any real competition.

A recent example of that would be jimmy23013's mad golfing session to shorten his self-matching regex. There is currently no competition on that challenge, but the desire to find an even shorter solution has been enough to come up with ever new approaches (5 or 6 and counting), and explore different solutions, which has also led to a much more elegant solution in the end. If the challenge hadn't been code golf, I'm pretty sure the answer would have remained as the first working solution.

Although the code golf aspect (as an example, since it's our most common challenge type) is often orthogonal to the actual problem at hand, it's what drives us to think outside the box and look into new approaches to the given problem, because "just any solution" isn't good enough.

If I compare this to Project Euler, which is essentially a site that hosts similar (although harder) programming challenges without a scoring/winning criterion, I always move on to the next problem once I've found a solution to one of them. Especially, on problems that happen to be just feasible with a naive approach, I don't learn anything from them, because there's no motivation to look at the problem from any possible angle.

Of course, not having any scoring at all is the extreme opposite. There's also the option of having subjective winning criteria as Dennis mentioned. But that isn't any better: if I can't directly measure the the quality of my solution, and compare it against other solutions on a fixed scale, how do I know that another approach will actually improve my solution? In the end, I'll still think "yeah this looks good, I hope the OP likes it better than the other solutions".

All in all, I think people who look down on code golf overestimate the importance of the specific winning criterion at hand. It's just that code golf works particularly well, because it's easy to measure, easy to compare, provides a wide variety of potential scores, and cannot be gamed (save for exploiting loopholes). But in the end, amazing code golf solutions aren't amazing because they're short, but because the author really looked into the problem from all angles and came up with a novel approach that blew everyone's mind. (This also means that the "best" (subjective) answer around here isn't always the shortest, but more often than not the striving for shortness is what inspired the answer.)

That said, I can understand if people would like to see other objective winning criteria being more prominent, or things like being easier to run on this site. And I'm all up for not being a pure code golf site, exploring new (objective) scoring systems, and doing more generic . But being objective is most important trait of any of those challenges, because without that, I'm sure many of us would lose the motivation to spend as much time on a single problem as we do sometimes.

\$\endgroup\$
  • 1
    \$\begingroup\$ This made it clear to me. Thank you. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 23:36
9
\$\begingroup\$

It is hard to discuss advantages and disadvantages of such a radical change; it completely changes the scope of the site. Right now, this site holds programming contests, meaning that we don't just try to solve a problem with a specific goal, but we compete against others that try to do the same.

Removing the objective winning criterion requirement and simply asking for "an efficient implementation" means that we're no longer looking at a competition, since there is no way to compare submissions. While that may be helpful to someone who is trying to solve a problem, solving people's problems is not what this site is about.

Re: disadvantages

[I]t hinders challenges that are looking for efficient solutions[.]

It shouldn't.

Yes, most plain questions require an additional effort from the OP (other just use an online platform for timing), but there are other options. We have , which means the competitor can time the submission himself, and , which requires no timing at all.

[I]t hinders challenges that are looking for [...] any solution at all.

That is by design, and not at all a problem in my opinion. Searching for any solution at all means that we don't have a competition anymore.

Yes, there are problems that are interesting and tough enough on their own without trying to come up with a particularly short or particularly fast solution, but just because something is interesting doesn't mean it should be on topic here.

Subjective winning criteria

Subjective winning criteria (i.e., author's choice) could be perceived as a middle ground between objective winning criteria and no winning criteria at all, but it's virtually no different from the latter.

If a competitor has several solutions, without knowing precisely what to aim for, he cannot choose which of his solutions he should submit. For the same reason, I'm not a big fan of challenges.

Simply letting the OP choose which answer should take the crown makes us no different from other programming Q&A sites. This isn't growth, it's losing our identity.

\$\endgroup\$
  • 1
    \$\begingroup\$ Thank you for your answer! Unfortunately, both alternatives have problems: golf-cpu is slow, and difficult for most users to participate in, and fastest-algorithm doesn't actually mean that the algorithm is fast. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 15:50
  • \$\begingroup\$ Simply letting the OP choose which answer should take the crown makes us no different from other programming Q&A sites I don't know why we need to be different (in this regard). I'm not proposing we lose objective goals (such as fast code), but proposing we lose objective scoring (measuring it) \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 15:51
  • 4
    \$\begingroup\$ 1. My point is, if the winning criteria we currently use to measure code speed are flawed, we should try to find better ones, not eliminate them. 2. Holding contests is not some minor difference between this site and, e.g., Stack Overflow, it is the main difference. 3. fast code isn't an objective goal. The same algorithm can be amazingly fast for some cases and excruciatingly slow for others. The same optimizations can be beneficial on some architectures and harmful on others. \$\endgroup\$ – Dennis Jan 17 '16 at 16:32
  • \$\begingroup\$ 1. I'm not talking about removing the current criteria. They are great. But we are restricting different possibilities of criteria because they can't be objectively scored. 2. I don't think so. We talking about programming challenges not programming contests. 3. Right, and the OP would take that into consideration when accepting an answer. Real code often takes efficiency into mind, and we don't always use benchmarks to write efficient code. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 16:57
  • 3
    \$\begingroup\$ 2. Google defines challenge as a call to take part in a contest or competition, especially a duel. 3. The OP would make an arbitrary decision, depending on what he thinks is the most efficient code. That's not an objective goal at all. \$\endgroup\$ – Dennis Jan 17 '16 at 17:09
  • \$\begingroup\$ 2. Regardless of semantics, a site that posts challenges like Project Euler is what makes us different (not that we score them), IMO. 3. When answering a question, I would write code that I think is fast, and the OP would accept answers that he thinks is fast. This is no different than any other site: I post what I would solve his problem the best, and he accepts the one that he thinks solves his problem the best. \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 17:15
  • 2
    \$\begingroup\$ @NathanMerrill 3. I assume by "any other site" you're talking about the other SEs. All of them are Q&A sites. PPCG is not. \$\endgroup\$ – Martin Ender Jan 17 '16 at 17:18
  • \$\begingroup\$ @MartinBüttner I disagree. We simply word them differently. Instead of asking "How do I write X in the smallest amount of code", we simply say "Write X in the smallest amount of code" \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 17:22
  • 1
    \$\begingroup\$ The difference between PPCG and actual Q&A sites lies precisely in the winning criteria we require. Take that away and, yes, you get yet another Q&A site. \$\endgroup\$ – Dennis Jan 17 '16 at 17:27
  • \$\begingroup\$ I agree. This change would bring us closer to the mold of a Q&A site. We would still be very different with "code challenges" (as I've been describing them), than with "code contests". \$\endgroup\$ – Nathan Merrill Jan 17 '16 at 17:34
  • 3
    \$\begingroup\$ Dennis, I disagree. Even without strict winning criteria, there is a very big difference between posting a question about something you're stuck with and writing a challenge others enjoy. There would also still be big differences between the kinds of answers you'd get. @NathanMerrill, you say that like becoming more of a Q&A site is something desirable. While the software is clearly meant to be used for Q&A sites, that does not mean that our community would actually work better by embracing its Q&A roots. (I think if would actually work better by SE acknowledging that we're not a Q&A site.) \$\endgroup\$ – Martin Ender Jan 17 '16 at 20:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .