6
\$\begingroup\$

Our FAQ on "on-topic questions" currently states:

All questions on this site, whether a programming puzzle or a code golf, should have ...

  • A clear specification of what constitutes a correct submission. Test cases are highly encouraged.

There have recently been a lot of questions which push this requirement. For example:

For all of these, whether or not an answer meets the specification is subject to debate. (It may not be coincidence that these are all ).

Should we close as off-topic any question for which it is not possible to write test cases?

\$\endgroup\$
7
  • 1
    \$\begingroup\$ Can you clarify "test cases?" For example, if a challenge gives example inputs, but has a subjective measure for "best" output, would that qualify? What about this question? \$\endgroup\$
    – PhiNotPi
    Commented Dec 30, 2013 at 14:07
  • \$\begingroup\$ @PhiNotPi, I think it might be possible to squeeze that question through by means of an error bound on the sum of squared errors when given extremely simple images, but it might be more honest to say that it's a good question which would probably fall foul of this proposal and require an exception to be made. \$\endgroup\$
    – Peter Taylor
    Commented Dec 30, 2013 at 14:14
  • \$\begingroup\$ I think going to close as off-topic goes a bit beyond the idea (would have to close all popularity-contest). Test cases are encouraged - not inevitable. I still think that reasonable openness of the specification has its place on this site. Nevertheless, the first part A clear specification of what constitutes a correct submission is still neccessary for submissions. \$\endgroup\$
    – Howard
    Commented Dec 30, 2013 at 15:30
  • \$\begingroup\$ @Howard, I think some popularity contests have objective specs: it's their scoring that's inherently subjective. What I'm trying to pin down is what constitutes a clear specification. What are necessary or sufficient criteria for a problem which doesn't admit testing to be clearly specified? \$\endgroup\$
    – Peter Taylor
    Commented Dec 30, 2013 at 15:55
  • 2
    \$\begingroup\$ @Howard "goes a bit beyond the idea (would have to close all popularity-contest). " You say that like it's a bad thing... \$\endgroup\$
    – dmckee --- ex-moderator kitten Mod
    Commented Dec 30, 2013 at 18:38
  • 1
    \$\begingroup\$ I'll throw this out: any alleged spec which generates several of the same joke answer or multiple distinct joke answers is probably too loose. \$\endgroup\$
    – dmckee --- ex-moderator kitten Mod
    Commented Dec 30, 2013 at 18:39
  • \$\begingroup\$ @dmckee You can assume that I am also not a big fan of popularity-contest, but I appreciate an intelligent and clever puzzle even in that category. \$\endgroup\$
    – Howard
    Commented Dec 30, 2013 at 20:06

3 Answers 3

Reset to default
3
\$\begingroup\$

I almost agree. and should have specific specs and should include test cases if relevant. However, some tags should not be so specific. For example, s (a is a form of popularity contest) should have a general specs. However, this does not mean they should have no specs.

A few examples of what I'd call good specs:

For :

  • this question gives a test case and specific specs, making it clear what constitutes a good answer.
  • this question does not give a test case because test cases are irrelevant but has specific specs.

For :

  • this question has general specs. However, it is not something like "output HelloWorld". It forces answerers to think.

For :

  • this question gives a specific scoring method and specific specs.

I personally feel that this site is for puzzles or challenges (just look at the site name!). Popularity Contests and Code Trolling questions are fun every now and then, but they aren't usually puzzles. I feel a is a good question if and only if it can be changed into a good or . It is fine to (every now and then) have a fun question, but the focus of this site is on puzzles.

Also, I thought was for questions with scoring criteria such as number of characters / number of languages. It seems like is being misused. People are tagging questions as because they seem like challenging problems, not because the scoring method is different. Something should be done about this.

\$\endgroup\$
2
  • \$\begingroup\$ Regarding the last remark: the code-challenge tag description explicitely tells what that tag is about. I already removed the tag from many, many questions because it was there without cause. \$\endgroup\$
    – Howard
    Commented Jan 3, 2014 at 8:08
  • \$\begingroup\$ @Howard In that case, I'll get to work (I've seen several questions that tag as code-challenge just because they are challenging to answer. Also some that are just mis-tagged.) \$\endgroup\$
    – Justin
    Commented Jan 3, 2014 at 8:15
1
\$\begingroup\$

Perhaps instead of closing questions for which it is not possible to write test cases, we could close questions (new questions - there's no point going back through the old questions) that do not include a test case or two.

One problem that this could cause is quite a large increase in question size if the input is quite large (I'm thinking of some really good questions like, for example, Joey's Bridges and Tunnels) - in this case would it be acceptable to provide the test cases in an external file?

Adding test cases to a question shows that the questioner has put some thought into the question and hasn't just chucked it together in 2 minutes to try and gain a few extra points reputation.

\$\endgroup\$
-2
\$\begingroup\$

The rule does not says that it must have a test case, it does says only that it is highly encouraged.

In case of , in fact there is a clear specification of what constitutes a correct submission: It is a submission that do not do what the lazy OP asks, or do it in a really awful way. Since this specification is subjective, it is hard to write a test case. Yes, it is possible to be clear and subjective at the same time. So, it is still within this rule.

For the Y2K question, the situation is similar to .

Hence, these questions can't be close for not following those rules, in fact they do.

It is true that all those questions are all subjective, so it would be hard to define any winning criteria other than . In fact, if it is a popularity contest, we do not even need a test case. I don't see that by having a subjective winning criteria as a bad thing, in fact I see it as a good thing.

I strong oppose to close those questions and I think that this would be harmful for this site. This site is all about programming for fun (no one would be squeezing bytes in a golfscript code or programming in befunge for anything else), so what we need to do is to keep the site fun. If you impose a one-size-fits-all rigid rule for everything, closing whatever does not fits there, the result will simply be the death of this site.

I am not saying that we do not need rules. I am saying that they should not be rigid. The rules serves for organizing the site in order that every user here could enjoy it in a fair way, its purpose is not to avoid anyone enjoying anything by making everything narrow and boring.

\$\endgroup\$
8
  • \$\begingroup\$ To your first paragraph: I know that the rule doesn't say that the question must have a test case. My question is whether a question must be capable of having a test case. That's very different. \$\endgroup\$
    – Peter Taylor
    Commented Jan 2, 2014 at 17:13
  • \$\begingroup\$ My contention is that "A submission that do not do what the lazy OP asks" is not a clear specification: the vast majority of randomly generated syntactically valid programs would meet that criterion, so it's not specifying anything. \$\endgroup\$
    – Peter Taylor
    Commented Jan 2, 2014 at 17:14
  • \$\begingroup\$ I don't see any logical connection in "In fact, if it is a popularity contest, we do not even need a test case". The winning criterion is a way of selecting the best answer which meets the spec. The point of testability is being able to determine whether an answer meets the spec. \$\endgroup\$
    – Peter Taylor
    Commented Jan 2, 2014 at 17:17
  • \$\begingroup\$ @PeterTaylor For your first comment: The question purpose is to allow people to be creative, innovative and thinking out-of-the-box, and not to strictly follow a rigid set of rules that are guarded by some test cases. They asks for code that do NOT do something or do it in a bad way, like underhanded and like this question. \$\endgroup\$
    – Victor Stafusa
    Commented Jan 2, 2014 at 17:58
  • 1
    \$\begingroup\$ @PeterTaylor For your second comment: In case of code-trolling, yes, I could write a simple Hello World in any language that I want, and that would meet the spec, but would also get some downvotes for not being creative. Following a spec guarded with a test case is simply not the point here, the point is simply to get the most upvotes, hence the popularity-contest. \$\endgroup\$
    – Victor Stafusa
    Commented Jan 2, 2014 at 18:00
  • 1
    \$\begingroup\$ @PeterTaylor For your third comment: If a question is tagged with popularity-contest the winning criterion is not the best answer which meets the spec. The winning criterion is the answer with most upvotes (that is the definition of this tag). If the objective is to get upvotes, meeting a spec is either secondary or unimportant. \$\endgroup\$
    – Victor Stafusa
    Commented Jan 2, 2014 at 18:02
  • 1
    \$\begingroup\$ The purpose of the winning criterion is to define best. A popularity contest is won by the answer with most upvotes out of those answers which meet the spec. Otherwise the logical extrapolation of your position is that the empty program should win all code-golf questions because the objective is to write a short program, not to meet a spec. \$\endgroup\$
    – Peter Taylor
    Commented Jan 2, 2014 at 18:16
  • \$\begingroup\$ @PeterTaylor Ok, in this case, even a loose subjective spec do it. In code-trolling we have a few restrictions and an answer that does not meets it are not elligible for being accepted (e.g. Does not provides any code). If the winning criterion is "shortest non-empty code", then a zero-sized code is automatically not elligible. This way, as you said, in popularity contest the most upvoted answer which follows the rules is the best. You may interpret "meet the spec" as "follows the rules". But this have nothing to do with test cases and we are still fine with that. \$\endgroup\$
    – Victor Stafusa
    Commented Jan 2, 2014 at 18:27

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .