12
\$\begingroup\$

We recently had a rather short challenge without any test cases (EDIT: no longer true). This sparked some conversation in chat, as well as some comments concerning its lack of test cases.

As far as I can tell, we haven't formally set down any rules as to whether we need test cases or not. Therefore:

Should test cases be required on challenges?

(Obviously, certain types of challenges inherently don't have test cases, such as and , so I'm not talking about those)

\$\endgroup\$
2
  • 1
    \$\begingroup\$ A challenge has to be clear to remain open. That can mean any number of things, including test cases. But if the challenge is to print the square of a number, test cases are rather meaningless... \$\endgroup\$
    – Dennis
    Feb 24 '16 at 3:12
  • 1
    \$\begingroup\$ @Dennis I think we can agree that test cases shouldn't make up the spec of the challenge. That said, even for a square of a number, test cases such as -1, 0, and large numbers can be useful. \$\endgroup\$
    – Nathan Merrill
    Feb 24 '16 at 3:18
20
\$\begingroup\$

Require? No. Encourage? Certainly.

Test cases help clarify a challenge, but in the case of the challenge linked in the question, the task is so simple that test cases don't really add any substance or clarification.

What would we do if we did require test cases? Close challenges as unclear until they have test cases? That seems overly strict, particularly in situations such as this.

\$\endgroup\$
4
  • \$\begingroup\$ If we did require them, then yes, closing them would be the appropriate action. The reasoning would likely be something along the lines of "We want questions that are as high quality as possible, therefore, we have set a minimum bar that all questions need to pass". \$\endgroup\$
    – Nathan Merrill
    Feb 24 '16 at 3:25
  • 4
    \$\begingroup\$ IMO we have already set a rather daunting "minimum bar" for challenges, especially to new users. While I don't think test cases are ever a bad idea, I think their necessity is proportional to the complexity of the task. \$\endgroup\$
    – Geobits
    Feb 24 '16 at 3:43
  • 2
    \$\begingroup\$ "the task is so simple that test cases don't really add any substance or clarification" is that true though? In a language where I'd have to implement fractional powers myself, it would certainly be helpful to be able to check some results to make sure I've actually implemented it correctly. (Of course, I could just calculate correct results with another language, but then again for most sequence challenges I could just look them up in OEIS and we usually expect people to include some relevant terms of those in challenge itself as well.) \$\endgroup\$
    – Martin Ender
    Feb 24 '16 at 7:44
  • 3
    \$\begingroup\$ @MartinBüttner Fair point. I still think they shouldn't be required though. \$\endgroup\$
    – Alex A.
    Feb 24 '16 at 17:02

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .