I think an important thing to remember about neural nets is that they are basically a good way to overfit. So, yes, you can approximate any function because you have lots and lots of variables, but you shouldn't fool yourself that you are getting the same information as when you write a deterministic equation with a few variables ("with four parameters I can fit an elephant, with five I can make him wiggle his trunk"... Von Neumann). You can the the baseball but you don't yet know the physics.

I think skilled biological actors use this overfitting to get really good at things without knowing how things actually work.

NN are still cool, though.