Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I don't think you understood the statement you are quoting.

The manifesto is not in any way advocating that you use byte length as a substitute for string length. (It does argue that string length is not very commonly necessary, and has unclear semantics because of the multiple different definitions for "character", but these arguments are unrelated to the statement you quoted.)

Here's the point the author was making, which you missed. Take a non-BMP code point like '𝄞', which is U+1D11E. In UTF-16 this is represented by code units D834 and DD1E. If you try to use a "substring" operation to take the first "character" of this with a C#/.NET substring operation, you will get an invalid string, since D834 by itself is not a valid UTF-16 string.

> I want to work on top of an abstraction which lets me treat text as text.

If you think that UTF-16 will let you say string[i] and always get the i'th character, you are mistaken. That is one of the main points of the essay.

> Yes, their are cases where the abstraction will leak. But those cases are very far and few in between.

If you write your apps this way, then you don't really support Unicode, you just support the BMP, without combining characters.



One of the advantages of working in UTF-8 is that if your code is broken, you find out about it right away - as soon as someone enters an accented character. If your UTF-16 code is broken you might not find it in testing.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: