That is essentially correct. The subtlety is that the hidden layer still has a finite (but arbitrarily large) number of nodes, while there are an infinite number of inputs. The solution to this is that you can keep adding nodes until the space between the nodes is close enough to the function you are modelling.