My favourite way of looking at it is to imagine neurons in a neural net as analogue nand-gates.
The logic within a computer processor is made entirely with nand-gates, and a processor is able to compute any function. Nand-gates have 'functional completeness' and can implement any other gate (AND, OR, NOT, NOR), and any other high level construct with those functions.
Similarly, neurons in a neural net can output a 'NAND' function if set up correctly.
This provides a logical inference of computational completeness of neural-nets
The logic within a computer processor is made entirely with nand-gates, and a processor is able to compute any function. Nand-gates have 'functional completeness' and can implement any other gate (AND, OR, NOT, NOR), and any other high level construct with those functions.
Similarly, neurons in a neural net can output a 'NAND' function if set up correctly.
This provides a logical inference of computational completeness of neural-nets