2
\$\begingroup\$

I've been considering answering a question in Super Stack!, specifically via use of this interpreter. However, the interpreter fails to compile, because there are two mistakes in it:

  • it's incorrectly escaped (the '' in the interpreter source code is interpreted as italics in the markup dialect in use rather than as an empty string);
  • the routine for printing usage information (in response to invalid command line arguments) contains a typo, leaving off the trailing ' of a string, which of course prevents the program compiling.

As such, in order to be able to use the language, I need to correct the problems in the interpreter. The first problem is arguably not a big deal, because it's possible to look at the source code used to generate the markup and copy from there, instead; it's a problem with an entirely mechanical fix. However, the second problem is a bit larger, given that it requires changing the interpreter to run programs in it.

In this case, the change is completely obvious (add a ' at the end of the offending line), and clearly makes no changes to the languages semantics nor otherwise gives an advantage (other than being able to compete with the language at all). However, being able to modify interpreters without them counting as postdating the challenge is something that would clearly be exploitable if unrestricted.

As such, I think we need a policy on what sorts of edits can be made to an interpreter after a question is posted, whilst still allowing answers made using the new version of the interpreter to count as competing. (It's already been established that you can't edit the interpreter to make it match the specification, but the case in which the interpreter appears to have become corrupted while being placed online is a different issue.) So what should that policy be?

\$\endgroup\$
1

3 Answers 3

Reset to default
8
\$\begingroup\$

All changes to the interpreter make the answer non-competing

That's the only policy that is consistent with our established consensus that the interpreter defines the language.

Specifications written in a natural language cannot possibly be as clear as a definition by code; code either does something or it does not. While an interpreter that crashes no matter what you do with it probably doesn't match the author's intentions, it still doesn't give us certainty what the interpreter was meant to do.

The proposal to fix the interpreter with a minimal edit distance is:

  • Artificial. The shortest fix could be commenting out parts of or even the entire code. That's not very useful.

  • Difficult to verify. With a large enough interpreter, iterating over all changes with edit distance X or less won't be feasible.

  • Ill-defined. Several "fixes" will probably tie on the shortest edit distance.

  • Prone to abuse. The shortest edit distance could be anything, related or not to what the interpreter should do.

That's a lot of issues to correct something that isn't an issue. Our current consensus is unambiguous. While it is annoying to have to mark an answer as non-competing because of a bug, it's also not a big deal.

\$\endgroup\$
6
  • \$\begingroup\$ This case is intended to fix typos that prevent the interpreter running, meaning that edit distance is entirely non-arbitrary (it's the same measure that, e.g., spell checkers use to figure out what the original, typoed word was!). In the interpreter in question, the problem is entirely a missing end-of-string marker on the version of the interpreter that was posted online (note: presumably the original, non-online, interpreter didn't have the typo in question). It is obvious that there's only one way to fix it, and edit distance formalizes that. \$\endgroup\$
    – user62131
    Dec 10, 2016 at 19:32
  • \$\begingroup\$ Actually, you're right in that commenting out the offending line would also work. But in this case, the resulting interpreter runs the program identically either way, because the line in question is never actually executed! As far as I can tell, that implies that it's completely objective what the interpreter is meant to do. \$\endgroup\$
    – user62131
    Dec 10, 2016 at 19:39
  • 1
    \$\begingroup\$ I can see how your proposal makes sense for that specific interpreter (after all, you only have to click View source to see the intended version), but that doesn't make it a good general policy. Even in this case, if line!='0' would be a fix with minimal edit distance. Re spell checking: Comparing a single word and an entire interpreter are two different pair of shoes. And even with single words, spell checkers make mistakes. \$\endgroup\$
    – Dennis
    Dec 10, 2016 at 19:42
  • \$\begingroup\$ You're looking at the wrong line. I consider the if line !=] to be simply an artifact of MediaWiki escaping (the interpreter available via "view source" is clearly just as valid as the interpreter that's on the rendered version of the page). The line that's causing the trouble here, though, is print 'superstack.py my_program.ss! [-d] in the usage() function (which never even runs when the interpreter is used to interpret a program). \$\endgroup\$
    – user62131
    Dec 10, 2016 at 19:45
  • 1
    \$\begingroup\$ Right. My point still stands though. Just because the shortest edit distance produces something sensible in this case doesn't make it a good general policy. \$\endgroup\$
    – Dennis
    Dec 10, 2016 at 19:51
  • \$\begingroup\$ I just wanted to point out that with our new changes to non-competing policy this answer is no longer accurate. You might want to leave a note in its body to indicate so. \$\endgroup\$
    – Wheat Wizard Mod
    Aug 6, 2017 at 1:15
0
\$\begingroup\$

Bugs can be corrected if they prevent the interpreter from running at all

If an interpreter crashes regardless of the input that it's given, it's acceptable to edit the interpreter in order to remove the crash. In order to prevent abuses, the edit must be the smallest possible edit in terms of Levenshtein distance that enables the interpreter to run at least one program without crashing.

\$\endgroup\$
7
  • \$\begingroup\$ I already asked this, and the answer was no, even assuming good faith. \$\endgroup\$
    – Fatalize
    Dec 10, 2016 at 18:24
  • \$\begingroup\$ This situation's a bit different. In the linked answer, the problem is that the interpreter doesn't match the spec, but you can treat it as defining a new spec. In the case that the interpreter doesn't work at all, it's clearly impossible for it to specify a language in the first place. \$\endgroup\$
    – user62131
    Dec 10, 2016 at 18:28
  • 1
    \$\begingroup\$ Well then if your fix it it's like creating a language after the challenge was posted, since the bugged interpreter doesnt specify a language. \$\endgroup\$
    – Fatalize
    Dec 10, 2016 at 18:34
  • \$\begingroup\$ If you disagree with this answer, why don't you post your own competing answer and see what the votes on it are like? That's normally the best way to solve disputes on Meta. \$\endgroup\$
    – user62131
    Dec 10, 2016 at 18:50
  • \$\begingroup\$ I don't disagree with it. I just have had multiple experiences of people choosing extremely stiff rules on this site so I'm just exposing the kind of arguments they told me. \$\endgroup\$
    – Fatalize
    Dec 10, 2016 at 18:57
  • 2
    \$\begingroup\$ I'd argue if an interpreter can't be run at all, then its not an interpreter in the first place. \$\endgroup\$
    – Nathan Merrill
    Dec 10, 2016 at 22:53
  • \$\begingroup\$ @NathanMerrill: Well, I assume it was an interpreter that existed offline, and something went wrong while posting it online, so that all we can see is a corrupted copy. \$\endgroup\$
    – user62131
    Dec 10, 2016 at 23:03
0
\$\begingroup\$

Interpreters that don't work at all shouldn't be too common. It's likely either the interpreter isn't finished, so there won't be an easy fix anyway, or it did work but was broken for some reasons.

In most cases, the interpreter doesn't work for reasons such as the grammer it is in has changed, or a library it has used was abandoned. We shouldn't force ourselves to install a lot of old softwares for that. We may just fix them in a way we think it should match the original behavior, unless someone proves otherwise.

In your case, if the interpreter worked if those problems are fixed, it's very likely a working interpreter has existed, but something happened when the author was posting it.

As nobody would disagree about what the original interpreter was, we don't need to be too technical. But if it is unclear, and the unclear part may change the validity of your submission, it's better not to use it or to mark it non-competing, even if the fix is easy.

\$\endgroup\$