I think the problem is that we can't write down all our assumptions, even if we try REALLY hard. This is some of what inspired the mathematical formalism movement; trying to make sure that we know what we think we know.