At what point do we start classifying multi layered neural networks as deep neural networks or to put it in another way 'What is the minimum number of layers in a deep neural network?'
$\begingroup$
$\endgroup$
2
-
$\begingroup$ One hidden layer is a deep network $\endgroup$– HughCommented Aug 13, 2016 at 13:17
-
6$\begingroup$ Minimum number of sand grains in a heap of sand. $\endgroup$– amoebaCommented Aug 18, 2016 at 22:39
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
"Deep" is a marketing term: you can therefore use it whenever you need to market your multi-layered neural network.
answered Aug 13, 2016 at 13:11
$\begingroup$
$\endgroup$
"Deep"
One of the earliest deep neural networks has three densely connected hidden layers (Hinton et al. (2006)).
"Very Deep"
In 2014 the "very deep" VGG netowrks Simonyan et al. (2014) consist of 16+ hidden layers.
"Extremely Deep"
In 2016 the "extremely deep" residual networks He et al. (2016) consist of 50 up to 1,000+ hidden layers.
$\begingroup$
$\endgroup$
As per the literature,
Schmidhuber, J. (2015). "Deep Learning in Neural Networks: An Overview". Neural Networks. 61: 85–117. arXiv:1404.7828free to read. doi:10.1016/j.neunet.2014.09.003.
https://en.wikipedia.org/wiki/Deep_learning
It is said that
There is no universally agreed upon threshold of depth dividing shallow learning from deep learning, but most researchers in the field agree that deep learning has multiple nonlinear layers (CAP > 2) and Schmidhuber considers CAP > 10 to be very deep learning
A chain of transformations from input to output is a Credit Assignment Path or CAP. For a feedforward neural network, the depth of the CAPs, and thus the depth of the network, is the number of hidden layers plus one.
