Has the family of models encapsulated by neural networks been studied from a functional analysis perspective? Are there any general theorems, results, etc. about neural networks from this perspective?
2
-
$\begingroup$ en.wikipedia.org/wiki/Universal_approximation_theorem $\endgroup$– Alex R.Commented May 26, 2016 at 23:30
-
$\begingroup$ neuralnetworksanddeeplearning.com/chap4.html $\endgroup$– Alex R.Commented May 26, 2016 at 23:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Probably the core appeal of neural networks is the so-called "universal approximation theorem". Loosely stated, it's possible to use a neural network to approximate a continuous function on a compact subset of $\mathbb{R}^n$ using only a single hidden layer and finitely many neurons.
The continuous and compact portions of that statement are the immediately obvious caveats. Other caveats include those related to actually training the neural network: it may not be easy to train achieve a specific precision because training neural networks is hard.
More information can be found in Chapter 4 of Neural Networks and Deep Learning by Michael Nielsen. Wikipedia has a formal statement of the theorem.
answered Jul 11, 2018 at 20:52