What character encodings may a submission use?

Question

In code-golf questions, we typically score submissions by their byte count. However, whenever a submission uses a character outside of the ASCII range, the question of which encoding the code uses comes up.

One language for which this is quite relevant is APL, due to its use of many characters outside of ASCII and which have multiple-byte UTF-8 encodings. It is also relevant in many other languages.

As far as I can tell, we don't have a policy on how to handle this currently.

Here are some possible ways to handle it:

• Everything is UTF-8.

• Languages may have specific encodings. If there is no standard encoding for a given language, use UTF-8.

• Submissions may have specific encodings. If none is given, a language-wide default applies. If the language has no default, UTF-8.

I'm sure there are other possibilities, and I would be interested in hearing proposals. I do believe this is an issue we need a firm policy on, because this question seems to come up every single time someone writes an answer in APL, and reasonably often on other answers as well.

Answer 1

Answer, else language, else UTF-8

I prefer the system where an answer may specify its encoding, or default to its langauge's standard encoding, and a language's standard encoding is assumed to be UTF-8 unless specified elsewhere.

This has a couple of nice properties:

• An answerer can use whatever encoding they want for a particular answer.

• Newer users can do the 'obvious thing', such as count characters in APL, and still be correct almost all of the time.

• Every answer has a well-defined byte count.

The one major drawback is the need to maintain a list of language-by-language defaults, but there are only a small number of commonly used languages that don't use UTF-8 (APL uses the APL code page, Pyth uses ISO-8859-1, etc.), so it shouldn't be that hard.

EDIT: To clarify something with regards to @Ypnypn's comment: Answer-defined encodings would be subject to the same standard loopholes as programming languages, namely that the encoding must have been specified before the question was asked, and specifying an enormous list of encodings, in the style of MetaGolfscript, is not allowed either.

Answer 2

Standard language encoding, else UTF-8

Answers should use the most appropriate encoding for the language, not whatever is convenient for the answer. If no standard encoding for the language exists, the default should be UTF-8.

We could compile a list of languages that require a specific encoding and have that list on meta for reference. If the language is not on the list, assume UTF-8.

Answer 3

Burden of demonstration

[edited]

Allowing charsets that are actually accepted in practice seems entirely reasonable, with some variant of Unicode as a fallback.

However I'm concerned that someone unfamiliar with the language in question may find it difficult to confirming the existence of a custom code page, or more importantly, to prove the non-existence of one.

If a language has a more efficient encoding than UTF-8 or UCS-2, it would be help to at least mention this in an answer, rather than assume that everyone else knows which encodings are accepted by some compiler or interpreter. (A link to a code page would be helpful but need not be a formal requirement.)

This should be only a trivial impost on the claimant, compared with the effort by a reader to prove or disprove a tacit assumption of a codepage.

Why am I raising this?

I've recently noticed a few answers that consist almost entirely of non-ASCII characters. For example in #266049 How many umbrellas to cover the beach? there are answers in Charcoal and 05AB1E claiming 26 and 13 bytes respectively.

When I queried the byte count for the 05AB1E entry, there was a response that cited a custom code page, so that would seem to be valid, but it doesn't seem reasonable that a casual reader should be expected to know this.

As far as I can see Charcoal is built atop Python 3, which means it implicitly assumes UTF-8, which means the claim of 22 bytes is unsupportable by any of the metrics in this page. (OK, it turns out Charcoal does have a code page, despite me looking and not finding one.)

Other thoughts

Ideally it would be obvious where to find the site FAQ, but when viewing a challenge and its submissions, I don't see anything that even suggests "site FAQ" or "site policy" documents, nor do I see anything referring to this "meta" site. (Maybe "meta" is some kind of standard thing on StackExchange, but if so, that in turn needs to be made clearer, by StackExchange itself.)

Is there even a FAQ document, or are these meta questions the only approximation of such a resource?