That's the thing. You don't have to replace every human in order for it to be scary. Current tech is perfectly capable of replacing 4 out of every 5 low-skilled workers, and that's enough to royally screw our current economic model.