> Consider the problem of naming a piece of music based on a short sample of the piece. That can be thought of as computing a function. Or consider the problem of translating a Chinese text into English. Again, that can be thought of as computing a function. Or consider the problem of taking an mp4 movie file and generating a description of the plot of the movie, and a discussion of the quality of the acting. Again, that can be thought of as a kind of function computation. Universality means that, in principle, neural networks can do all these things and many more.
Eesh, that seems like poor technical written, and potentially misleading. It's far from clear that these are computable functions in the first place; Penrose argues (see notably "Shadows of the Mind") that there are functions which can be computed by the human brain that are undecidable over the Church-Turing thesis.
"Just because we know a neural network exists that can (say) translate Chinese text into English, that doesn't mean we have good techniques for constructing or even recognizing such a network." seems very hand wavy - we don't know that such a neural network exists without first going through a big set of still controversial assumptions.
The rest of the article is good, but I'm not a big fan of that section. It's not a huge part, but I think this could give some students the wrong idea about what is a computable function, and what neural networks can compute.
By function I think he meant mapping input points to output points in an abstract plane.
So in that sense a piece of music, or a sentence in one language is a point of input, while name of music or sentence in another language is another point.
Everything is a function as long as there is a way to turn that thing into inputs that correspond to outputs.
Ok, so by your reasoning let's have a function as a point of input, and whether it halts or not as a point of output.
So now we have a function, I can't wait till "we have good techniques for constructing such a network" that maps those inputs and outputs in an abstract plane :)
That is a good example. But you are forgetting that neural networks are approximating the actual functions, so the function you described could be built with some kind of confidence level in the answer. Just like you can have some confidence that certain code will not halt from experience, neural network could also be built to do that. Not all possible functions though, unless you have infinitely large network with infinite computing power.
Yeah but it's still misleading as we are talking about approximating continuous functions here, not any function. Those examples are not clearly computable, or even just continuous..
I would agree that the examples can be somewhat misleading, but not because of computability. Universal approximation theorem for neural networks doesn't care about computability, but it does require that the domain of the function is finite (or to be more precise, compact).
For example, suppose that the function to be approximated is simply f(x) = x. For any real numbers a < b we can produce an approximation that works well for a ≤ x ≤ b, but it cannot be a good approximation for all real numbers x. (The sigmoid in the hidden layer means that every approximation has to be a bounded function, and a bounded function cannot be a good global approximation of f(x) = x.)
Therefore, the translation example works if we assume that the number of different Chinese texts is finite, but otherwise nothing is guaranteed.
Eesh, that seems like poor technical written, and potentially misleading. It's far from clear that these are computable functions in the first place; Penrose argues (see notably "Shadows of the Mind") that there are functions which can be computed by the human brain that are undecidable over the Church-Turing thesis.
"Just because we know a neural network exists that can (say) translate Chinese text into English, that doesn't mean we have good techniques for constructing or even recognizing such a network." seems very hand wavy - we don't know that such a neural network exists without first going through a big set of still controversial assumptions.
The rest of the article is good, but I'm not a big fan of that section. It's not a huge part, but I think this could give some students the wrong idea about what is a computable function, and what neural networks can compute.