You do not "read" binary from right to left. At least in English, the most significant digit is on the left, just like any other base (e.g. base 10, base 8, or base 89432890432).
Also, for your leading 0, I would write: Bob has 23 apples, and I have 5. I wouldn't write: Bob has 23 apples, and I have 05. Base 2 is not special, so I wouldn't write 01.
You can also turn your order argument on its head: since the order of digits in base 10 is exactly the same as base 2 (most significant on the left), you could say that "As easy as 1, 2, 3" should actually be read as "As easy as 3, 2, 1", which matches your binary reading "As easy as 3, 2, 1".
The two can be melded together: There are 10 kinds of people in this world, those who understand binary, those who don't, and those who suffer off-by-one errors.
I thought it was: There are 10 kinds of people in this world, those who understand binary, those who don't, and those who were expecting a base 3 joke.
There are 10 kinds of people in the world. Those who know binary, those who know ternary, those who know base four, ..., and those who know that every base is base 10.
