19
\$\begingroup\$

In code-golf questions, we typically score submissions by their byte count. However, whenever a submission uses a character outside of the ASCII range, the question of which encoding the code uses comes up.

One language for which this is quite relevant is APL, due to its use of many characters outside of ASCII and which have multiple-byte UTF-8 encodings. It is also relevant in many other languages.

As far as I can tell, we don't have a policy on how to handle this currently.

Here are some possible ways to handle it:

  • Everything is UTF-8.

  • Languages may have specific encodings. If there is no standard encoding for a given language, use UTF-8.

  • Submissions may have specific encodings. If none is given, a language-wide default applies. If the language has no default, UTF-8.

I'm sure there are other possibilities, and I would be interested in hearing proposals. I do believe this is an issue we need a firm policy on, because this question seems to come up every single time someone writes an answer in APL, and reasonably often on other answers as well.

\$\endgroup\$
1
  • 4
    \$\begingroup\$ "As far as I can tell, we don't have a policy on how to handle this currently." That's not entirely correct. The code golf tag wiki says "Unless the question is specified to be scored by characters, it is scored by bytes. If it doesn't specify a character encoding to use for scoring, answers which use Unicode code points outside 0 to 255 should state the encoding used." \$\endgroup\$
    – Martin Ender
    Sep 3, 2015 at 11:57

2 Answers 2

Reset to default
25
\$\begingroup\$

Answer, else language, else UTF-8

I prefer the system where an answer may specify its encoding, or default to its langauge's standard encoding, and a language's standard encoding is assumed to be UTF-8 unless specified elsewhere.

This has a couple of nice properties:

  • An answerer can use whatever encoding they want for a particular answer.

  • Newer users can do the 'obvious thing', such as count characters in APL, and still be correct almost all of the time.

  • Every answer has a well-defined byte count.

The one major drawback is the need to maintain a list of language-by-language defaults, but there are only a small number of commonly used languages that don't use UTF-8 (APL uses the APL code page, Pyth uses ISO-8859-1, etc.), so it shouldn't be that hard.

EDIT: To clarify something with regards to @Ypnypn's comment: Answer-defined encodings would be subject to the same standard loopholes as programming languages, namely that the encoding must have been specified before the question was asked, and specifying an enormous list of encodings, in the style of MetaGolfscript, is not allowed either.

\$\endgroup\$
7
  • 2
    \$\begingroup\$ Personally I think giving individual answers the opportunity to use any encoding opens up a can of worms. One example is Antiferromagnetic Ordering, where many folks used unicode up- and down-arrows, which are 3 bytes each. These are also part of the APL code page, and are thus 1 byte each for APL. Per this meta answer, answers to that challenge may try to claim the EBCDIC-based APL encoding when in fact it's unnatural for their language to do so. \$\endgroup\$
    – Alex A.
    Sep 3, 2015 at 16:08
  • 9
    \$\begingroup\$ @AlexA. Is that a problem? \$\endgroup\$
    – Martin Ender
    Sep 4, 2015 at 14:15
  • 3
    \$\begingroup\$ So I can make a custom encoding for each answer, which "happens" to use 0.5 bytes for each character in the solution? -1. \$\endgroup\$
    – Ypnypn
    Sep 6, 2015 at 19:19
  • 3
    \$\begingroup\$ @Ypnypn no, you can use any existing encoding, (the same for languages: you can use any existing language) \$\endgroup\$
    – edc65
    Sep 6, 2015 at 19:33
  • 2
    \$\begingroup\$ I like the idea of being able to specify the preferred encoding. I'd especially like to use UTF-21 to encode text that primarily consist of non-BMP code points (which would otherwise be 4 bytes per character under both UTF-8 and UTF-16). ;-) \$\endgroup\$
    – C. K. Young
    Sep 7, 2015 at 3:50
  • \$\begingroup\$ This could be used for cheating, if the scored is measured in bytes. \$\endgroup\$
    – univalence
    Jul 17, 2016 at 11:29
  • 3
    \$\begingroup\$ You might want to mention explicitly that there must actually exist an interpreter/compiler which understands the source file in the chosen encoding (so you can't score your answer as ISO 8859-1 if the interpreter chokes on anything that isn't valid UTF-8). \$\endgroup\$
    – Martin Ender
    Sep 28, 2016 at 20:35
5
\$\begingroup\$

Standard language encoding, else UTF-8

Answers should use the most appropriate encoding for the language, not whatever is convenient for the answer. If no standard encoding for the language exists, the default should be UTF-8.

We could compile a list of languages that require a specific encoding and have that list on meta for reference. If the language is not on the list, assume UTF-8.

\$\endgroup\$
3
  • \$\begingroup\$ I'd be OK with this. \$\endgroup\$
    – isaacg
    Sep 3, 2015 at 16:31
  • 3
    \$\begingroup\$ It's fairly common practice (especially in kolmogorov complexity, or other challenges that require compression) to use extended ASCII (primarily ISO 8859-1), which means that you get the full 256 possible values for each byte. This answer would disallow that for most languages, which would have to use multi-byte characters for every value outside (7-bit) ASCII. \$\endgroup\$
    – Martin Ender
    Sep 4, 2015 at 14:15
  • 3
    \$\begingroup\$ I dislike this answer, because there are many languages that can use multiple character encodings (Python is one), and it doesn't clearly specify whether or not that would be valid. \$\endgroup\$
    – user45941
    Jun 24, 2016 at 4:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .