I used to feel similarly. Now I find the double space a visual distraction that doesn't in any way improve readability.

The effect of the double space is, I suspect, a product of the reader's expectations: if you expect it, its absence creates mental work, detracting from readability; if you don't expect it, its presence is what creates mental work.