2
\$\begingroup\$

I saw that in many golfing programming languages with a SBCS other than UTF-8 there is a converter (encoding and decoding) that converts the code/file from the SBCS defined characters to UTF-8 and vice versa.

What is the point of having the UTF-8 representation of the code bytes? My only guess is to be able to save the code in 1-byte characters with known charset for golfing score evaluating purposes. However, if it for saving the program in UTF-8 encoding, isn't control characters like EOF can impact on reading back the file?

Plus, golfing-wise, does all challenges consider SBCS as valid byte-based scoring? or there are challenges that consider only UTF-8 as a 1 byte?

Thanks.

\$\endgroup\$

1 Answer 1

7
\$\begingroup\$

The UTF-8 representation allows the code to be human-readable (to the extent that this is possible for a dedicated golfing language) while being able to be scored by "number of characters". Since our default is to count bytes, golfing languages have an incentive to use all 256 bytes. Having a printable representation for each of them is just convenient. Note that the interpreter still has to correctly run the corresponding byte-encoded program, and that program is what we're actually scoring.

More technically, here's how a typical golfing language uses its SBCS, and how TIO and this site accommodate it:

  • The interpreter has one internal representation for each of the 256 bytes.
  • The interpreter has (at least) two modes: it can take the program as a bytestring and parse each byte as its internal representation, or it can parse a UTF-8 string of characters on its SBCS and parse each character like the internal representation of the corresponding byte. Apart from this part of the parsing process, the modes should be functionally identical.
  • TIO displays and (as far as I know) runs the program as a UTF-8 string in UTF-8 mode, but displays its score as if it used the equivalent bytestring program in bytestring mode. We do the same. It's generally understood that the UTF-8 program represents the bytestring program. This is a compromise between correctness (the UTF-8 string itself is not what's scored) and convenience (no-one wants to read already cryptic programs as hexdumps).
\$\endgroup\$
3
  • \$\begingroup\$ Thanks for the answer. So, just for clarification, the interpreter takes the input from the user/file as SBCS, convert it to UTF-8, and then the lexer is reading and tokenizing the UTF-8 characters? (For example, the common used website TIO here, get the input from the user, encode the input to UTF-8 using the language converter, then execute it with the interpreter of that language?, And the score counting over there is just the length of characters in the input (since they are not really one-byte)). \$\endgroup\$
    – SomoKRoceS
    Jul 26, 2020 at 12:47
  • \$\begingroup\$ @SomoKRoceS See my edit. \$\endgroup\$
    – Zgarb
    Jul 26, 2020 at 14:35
  • \$\begingroup\$ Got it! Thanks :) \$\endgroup\$
    – SomoKRoceS
    Jul 26, 2020 at 14:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .