This is over my head. Why did we need to take a trip to discover why \n is encoded as a byte with the value 10? Isn't that expected? The author and HN comments don't say, so I feel stupid.
The point is to ask "who" encoded that byte as the value of 10. If you're writing a parser and you parse a newline as the escape sequence `\n`, then where did the value 10 come from? If you instead parse a newline as the integer literal `10`, then where does the actual binary value 1010 come from?
The ultimate point of this exercise is to alter your perception of what a compiler is (in the same way as the famous Reflections On Trusting Trust presentation).
Which is to say: your compiler is not something that outputs your program; your compiler is also input to your program. And as a program itself, your compiler's compiler was an input to your compiler, which makes it transitively an input to your program, and the same is true of your compiler's compiler's compiler, and your compiler's compiler's compiler's compiler, and your compiler's compiler's compiler's compiler's compiler, and...
It is a kind of bug/quirk: it implies that the compiler is not buildable from scratch in its current form. It depends on a binary that knows how to translate \n.
The earliest reference on my shelf is Kernighan and Ritchie, 1978. Like you say, it's probably just been passed on from generation to generation. And it's easier to say "just like C" than to make up a new convention. Python uses the same convention.
If you had to rebuild the rust compiler from scratch, and all you had was rustc's source code, there's nothing in the source code to tell you what '\n' actually maps to.
It's an interesting real-world example of the Ken Thompson hack.
The thing is, why 10? Why not 9 or 11? The code says "if you see 'string of newline character', output 'newline character'". How does the compiler know what a newline character is? Its code in turn just says "if you see 'string of newline character', treat it as 'newline character'"...
As a human I can just Google "C string escape codes", but that table is nowhere to be found inside the compiler. If C 2025 is going to define Start of Heading as \h, is `'h' => cooked.push('\h')` going to magically start working? How could it possibly know?
Clearly at some point someone must've manually programmed a `'n' => 10` mapping, but where is it!?
> The thing is, why 10? Why not 9 or 11? The code says "if you see 'string of newline character', output 'newline character'". How does the compiler know what a newline character is? Its code in turn just says "if you see 'string of newline character', treat it as 'newline character'"...
You can look into the ascii table then.
10 is the line feed.
'\n' is there to help human because it makes more sense than '10'.
Nobody is asking why 'a' is equal to '61'.
From the codebase, you know that '\n' is a char. A char is a value between 0 and 255, if you explicitly convert '\n' to int then you happen to find the ascii value and you are good to go and there is no need to pretend there is any poetry in this.
Requoting you:
> "if you see 'string of newline character', output 'newline character'"
It simply becomes "if a you see 'the arbitrary symbol for the new line', output 'the corresponding ascii value'".
I read the quote as "if you see 'a', output 'a' in ascii code." which is not mysterious in any kind of way.
IMO, the article does not make sense at all because it pretends to wonder where an hexadecimal value is coming from, but with any other kind of symbol it would be the exact same article, you enter 'a' and find some weird hexadecimal value and you won't be able to correctly trace it from the source code.
It would make sense it 'a' would display 'a' but '\n' would display some hexadecimal garbage in a text editor.
> From the codebase, you know that '\n' is a char. A char is a value between 0 and 255, if you explicitly convert '\n' to int then you happen to find the ascii value and you are good to go and there is no need to pretend there is any poetry in this.
But how does the computer know which int to output when you "explicitly convert '\n' to int"? As humans, we can clearly just consult the ASCII table and/or the relevant language standard, but the computer doesn't have a brain and a pair of eyes, instead it must store that association somewhere. The purpose of this article is to locate where the association was originally entered into the source code by some human.
The question is less interesting for ordinary characters like 'a', since the codes for those are presumably baked into the keyboard hardware and the font files, and no further translation is needed.
> The question is less interesting for ordinary characters like 'a', since the codes for those are presumably baked into the keyboard hardware and the font files, and no further translation is needed.
It's true that the question is less interesting for regular characters, but your explanation why is way off base.
Consider a computer whose only I/O is a serial console. It is concerned with neither a keyboard nor a font file.
For a computer whose only I/O is a serial console, I'd say that it has no character 'a', but only a character 0x61 with such-and-such properties and relationships with other characters. It's when we press the 'A' key on our keyboard and get 0x61 out, or put 0x61 in and see the glyph 'a' on a display or a printout, that the code becomes associated with our human concept of the letter.
That is, suppose I design a font file so that character 0x61 has glyph 'b' and 0x62 has glyph 'a', and I accordingly swap the key caps for 'A' and 'B' on my keyboard. If I write a document with this font file and print it off, then no one looking at it could tell that my characters had the wrong codes. Only the spell-checker on my computer would complain, since it's still following its designers' ideas of the character codes 0x61 and 0x62 are supposed to mean within a word.
I understand and share the excitement on this subtle topic, but this only exists on a source code level. There’s a list of source codes linked in time by compilation processes that eventually lead to a numeric literal entered by a human.
But physical computers knew what to insert in place immediately, because there was 0x0a somewhere in binary every time.
Of course our physical computers know what to insert, since it was embedded in the binary. But it hasn't always been embedded "every time": there was a point in the past where someone's physical computer didn't know what to insert, and so they had to teach it by hand. Without the source code (or some human-entered code) at the end of the chain, we'd have to insist that the code was embedded from the very dawn of time, which would be rather absurd. Personally, I like how this article and other such projects push back on some of the mysticism around bootstrapping that sometimes floats around.
You're operating on different levels. Of course we know ASCII 10 is newline. The comment you're replying to is asking: yes but how does the CPU know that, when it runs the compiler? Obviously it sees the number 10 in the machine code. Where did that 10 come from - what is the provenance of that particular byte? It didn't come from the source code of the compiler, which just says \n, and it didn't come from the ASCII table, because that's just a reference document for humans, which the computer doesn't know about.
Here's another way to think of the inspiration for the article. You're creating a file to use as input to another computer program (in this case, rustc). Your text file contains the ascii strings 'a', and '\n'. The rustc computer, when reading the text file, reads the corresponding byte sequences - 39, 97, 39, and 39, 92, 110, 39, respectively. The first byte sequence contains the 97 that you desire, but the second sequence does not contain a 10. Yet, rustc somehow knows to generate a 10 from 39, 92, 110, 39. How?
Check the first Unicode codepoints, 10 is defined there:
000A is LINE FEED (LF) = new line (NL), end of line (EOL)
It was already 10 in ASCII too (and the first 128 codepoints of Unicode are mostly the same as ASCII [I think there are a few tiny differences in some control characters]).
So to answer your question: it's neither 9 nor 11 because '\n' stands for "new line" and not for "character tabulation" nor for "line tabulation" (which 9 and 11 respectively stands for).
> Clearly at some point someone must've manually programmed a `'n' => 10` mapping
