It's my understanding from watching the DeCSS fiasco unfold that the "illegal number" phrase was coined to mock the DMCA's criminalization of circumvention tools, and the first illegal numbers described were various representations of DeCSS. I don't think that position is unreasonable or wrong headed at all.
You are right that, when applied to actual copyrighted materials, the only obvious options are A: "We'll voluntarily submit to the legal fiction that bits have color (or at least shades of gray) in the form of copyright law, because we can see some benefit to doing so," and B: "We believe that a society whose rules can be fully justified by concrete logic, evolutionary morality, and the laws of physics is preferable to a society founded upon legal fictions, and abolition of the concept of intellectual property follows from this."
However, I believe a good number of people hold position C: "We believe copyright law has significantly overstepped its useful bounds, and will pretend to hold position B in order to achieve a reasonable compromise."
Even your use of the phrase "... the legal fiction that bits have color ..." sounds silly to me. Compare that phrase to "... the legal fiction that bicycles have an invisible ownership property ..." to see why. The entirety of the law is about similar fictions.
The idea that someone owns the part of my brain that has been indelibly imprinted with their copyrighted expression sounds silly to me.
If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of every one, and the receiver cannot dispossess himself of it. Its peculiar character, too, is that no one possesses the less, because every other possesses the whole of it. He who receives an idea from me, receives instruction himself without lessening mine; as he who lights his taper at mine, receives light without darkening me. -Thomas Jefferson
Ideas are fundamentally different from bicycles. One cannot share a bicycle indefinitely without each user having proportionally less access. One can share bits and ideas. Show me where in silicon the color of a bit is stored. The entire concept of colorful bits is a fabricatiion of the human imagination, not a fundamental property of the universe. The atoms comprising a bicycle, on the other hand, can indeed be under the control of the atoms comprising a human being, to the physical exclusion of others.
Ownership of a bicycle isn't about physically controlling the bicycle. It's about the path of the bicycle through space-time, which is exactly how color operates -- it tracks the flow of information through space-time. There is no physical aspect of a bicycle that controls ownership. It's color, just like with bits.
I'm sure we both understand well enough the trade-offs that society makes in granting copyrights. Reasonable people have differing opinions, but these trade-offs existed long before computers, which have only shifted the balance of the bargain. The fact that bits encode information as numbers is irrelevant to the discussion.
You are right that, when applied to actual copyrighted materials, the only obvious options are A: "We'll voluntarily submit to the legal fiction that bits have color (or at least shades of gray) in the form of copyright law, because we can see some benefit to doing so," and B: "We believe that a society whose rules can be fully justified by concrete logic, evolutionary morality, and the laws of physics is preferable to a society founded upon legal fictions, and abolition of the concept of intellectual property follows from this."
However, I believe a good number of people hold position C: "We believe copyright law has significantly overstepped its useful bounds, and will pretend to hold position B in order to achieve a reasonable compromise."