23
\$\begingroup\$

TL;DR

Look at this: 😀. It's an emote. Found most commonly in it's natural enviroment the iPhone, it looks to be a harmless character.

Until you realize the dastardly thing is really two characters: \ud83d\ude00. Uh-oh.

But try to select it. On any sane browser, it is treated as a single character. This leads me into my question. What is a character?

More

There are two conventional systems of scoring in . Most often is the byte count, the number of UTF-8 (or whatever encoding it uses) literal bytes the code contains. The second is that of character counts, the… what?

I've always thought that a character was a single entity of selectability. However, after playing around with emoticons, I realize that they are often two+ character sequences.

In such challenges where characters are counted, what shall an emoticon be considered? It seems to be the most logical choice to count them as two, when they are indeed two characters, however, by the intuitive definition of a character, they would be counted as one.

What is the definition of a character, by our standards?

\$\endgroup\$
1
  • 6
    \$\begingroup\$ This is actually a big problem on Mac OS X terminal - it considers it two characters, but Unicode says that it's one, so the interface glitches out. It breaks nano, vim, and the command line. \$\endgroup\$
    – Addison Crump
    Commented Dec 20, 2015 at 23:04

3 Answers 3

Reset to default
32
\$\begingroup\$

A character is a character.

😀 is a single Unicode character, specifically U+1F600. UTF-16 happens to encode it using two surrogate characters, but that's not different from UTF-8 encoding it as four bytes.

\$\endgroup\$
13
  • 11
    \$\begingroup\$ And there's nothing natural about iPhones... ;) \$\endgroup\$
    – Dennis Mod
    Commented Dec 20, 2015 at 22:40
  • \$\begingroup\$ It might be of interest in this case to liken "characters" to graphemes. \$\endgroup\$
    – Alex A. Mod
    Commented Dec 21, 2015 at 0:16
  • \$\begingroup\$ If your encoding is UTF-8, then is it two characters? \$\endgroup\$
    – geokavel
    Commented Dec 21, 2015 at 18:01
  • 4
    \$\begingroup\$ @geokavel No, a single Unicode character is a single Unicode character, no matter which encoding is used. \$\endgroup\$
    – Dennis Mod
    Commented Dec 21, 2015 at 18:02
  • 1
    \$\begingroup\$ But, i thought 0x1F600 is too big a number to fit into a UTF-8 character. What of the fact that typing this into your address bar returns 2 javascript:alert("😀".length);? \$\endgroup\$
    – geokavel
    Commented Dec 21, 2015 at 18:06
  • 1
    \$\begingroup\$ UTF-8 can encode all characters up to U+FFFFFFFF, most of which are no longer used. UTF-16 (which is what JavaScript uses internally) uses two surrogate characters to encode that particular Unicode character. None of that changes the fact that it is a single character, no matter how it is encoded. \$\endgroup\$
    – Dennis Mod
    Commented Dec 21, 2015 at 18:10
  • 2
    \$\begingroup\$ Then there's also CESU-8, which encodes astral plane characters as surrogate pairs, then encodes each surrogate character using UTF-8. It's used in Oracle's database because Java strings are UTF-16 internally. The abbreviation means "compatibility scheme for UTF-16: 8-bit". \$\endgroup\$
    – John Dvorak
    Commented Dec 23, 2015 at 11:25
  • \$\begingroup\$ In ES6 they added a new syntax for higher characters that actually handles reversal and a couple of other operations properly. \$\endgroup\$
    – SuperJedi224
    Commented Dec 26, 2015 at 15:02
  • \$\begingroup\$ @geokavel The fact that Javascript thinks this character has length 2 is just just a bug in Javascript: mathiasbynens.be/notes/javascript-unicode (It comes from the fact that Javascript's length counts UTF-16 code units.) \$\endgroup\$
    – ShreevatsaR
    Commented Dec 13, 2016 at 1:35
  • \$\begingroup\$ @ShreevatsaR That's not a bug. string.length is defined as the number of code units in the string, not the number of characters. \$\endgroup\$
    – Dennis Mod
    Commented Dec 13, 2016 at 1:42
  • \$\begingroup\$ @Dennis Well it's a design bug in the definition of string.length then. (Isn't that what it means to talk of a bug in a programming language rather than an implementation: an instance of poor design/definitions?) Practically no one who has a string wants to count number of UTF-16 code units in it, rather than either number of abstract characters or number of bytes. \$\endgroup\$
    – ShreevatsaR
    Commented Dec 13, 2016 at 1:48
  • \$\begingroup\$ @ShreevatsaR The length of an indexable object should match the number of indices it has. Since "😀"[0] and "😀"[1] correspond to the surrogates that define the emoji, the number of indices is 2. I'm not saying that byte and character count would not be useful, but you need the number of code units as well. \$\endgroup\$
    – Dennis Mod
    Commented Dec 13, 2016 at 1:57
  • \$\begingroup\$ Point, but different ways of leaking code-units (like indexing on a string giving surrogates) is all the same bug IMO. As evidenced by the fact that ES6's Array.from(string) and for…of iterate over characters. They're making these fixes, because of not getting it right the first time. The changes in ES6 diminish the need to iterate by indices, which further diminishes the need to count indices. (Arguably the existence of surrogate pairs in the Unicode standard is itself a bug, leaking implementation details of code units (in a particular encoding, UTF-16) into the code-point abstraction.) \$\endgroup\$
    – ShreevatsaR
    Commented Dec 13, 2016 at 2:44
17
\$\begingroup\$

Nope, that's still 1 character, 4 bytes.

Unicode is weird.

See also: The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

\$\endgroup\$
3
  • 1
    \$\begingroup\$ 4 bytes in UTF-8. A custom code page could work differently. \$\endgroup\$
    – isaacg
    Commented Dec 22, 2015 at 9:15
  • \$\begingroup\$ @isaacg But surely it would be a different character if it was a different codepage... Just with the same glyph. \$\endgroup\$
    – wizzwizz4
    Commented Dec 22, 2015 at 9:39
  • \$\begingroup\$ @wizzwizz4 I believe isaacg meant different Unicode encoding, not a different code page. \$\endgroup\$
    – John Dvorak
    Commented Dec 23, 2015 at 11:30
4
\$\begingroup\$ 
$ printf 😀 | wc
       0       1       4
$

1 character and 4 bytes (as the other answers point out).

Certain applications (generally those dealing with fixed-width fonts, e.g terminals and text editors) choose to display such characters over the space of 2 "normal" characters.

But character rendering by specific apps has nothing to do with byte/character count, especially for the purposes of .

\$\endgroup\$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .