They can compute any function, but the same can be said about splines, polynomials and many other ways to approximate functions. The problem is with an algorithms which will not lead to overfitting and can really reasonably approximate any function.
Wavelets are my favorite approximate function to build up with.
But yeah, the overfitting problem / approximation is the real issue. I do like seeing the hybrid approaches (Genetic Algorithsm searching random Neural Nets with a little bit of backpropigation for some measure)
