I was convinced both of those quotes were from Dijkstra (as quotes go, the Mark Twain of CS).
Turns out, the second one is originally from one Phil Karlton (as reported by Tim Bray [1] , that's enough authority for me), without the "off-by-one" part.
The first one, which I was goddamned sure was Dijkstra, is actually by David (not A.) Wheeler (via Butler Lampson) [2], inventor of the subroutine.
That rabbit-hole took me longer than expected. Hopefully I'm saving some time for someone else ;)
My quote list has the second half as a later elaboration:
"All problems in computer science can be solved by another level of indirection"
"But that usually will create another problem."
-- David Wheeler
"...except for the problem of too many layers of indirection."
-- Kevlin Henney
You do not "read" binary from right to left. At least in English, the most significant digit is on the left, just like any other base (e.g. base 10, base 8, or base 89432890432).
Also, for your leading 0, I would write: Bob has 23 apples, and I have 5. I wouldn't write: Bob has 23 apples, and I have 05. Base 2 is not special, so I wouldn't write 01.
You can also turn your order argument on its head: since the order of digits in base 10 is exactly the same as base 2 (most significant on the left), you could say that "As easy as 1, 2, 3" should actually be read as "As easy as 3, 2, 1", which matches your binary reading "As easy as 3, 2, 1".
The two can be melded together: There are 10 kinds of people in this world, those who understand binary, those who don't, and those who suffer off-by-one errors.
I thought it was: There are 10 kinds of people in this world, those who understand binary, those who don't, and those who were expecting a base 3 joke.
There are 10 kinds of people in the world. Those who know binary, those who know ternary, those who know base four, ..., and those who know that every base is base 10.
"All problems in computer science can be solved by another level of indirection, except for the problem of too many layers of indirection."
Hence this level of indirection, the discrete "drive", is what he thinks should go poof.