14
$\begingroup$

At what point do we start classifying multi layered neural networks as deep neural networks or to put it in another way 'What is the minimum number of layers in a deep neural network?'

$\endgroup$
19
$\begingroup$

"Deep" is a marketing term: you can therefore use it whenever you need to market your multi-layered neural network.

$\endgroup$
14
$\begingroup$

"Deep"
One of the earliest deep neural networks has three densely connected hidden layers (Hinton et al. (2006)).

enter image description here

"Very Deep"
In 2014 the "very deep" VGG netowrks Simonyan et al. (2014) consist of 16+ hidden layers.

enter image description here

"Extremely Deep"
In 2016 the "extremely deep" residual networks He et al. (2016) consist of 50 up to 1,000+ hidden layers. enter image description here

$\endgroup$
9
$\begingroup$

As per the literature,

Schmidhuber, J. (2015). "Deep Learning in Neural Networks: An Overview". Neural Networks. 61: 85–117. arXiv:1404.7828free to read. doi:10.1016/j.neunet.2014.09.003.

https://en.wikipedia.org/wiki/Deep_learning

It is said that

There is no universally agreed upon threshold of depth dividing shallow learning from deep learning, but most researchers in the field agree that deep learning has multiple nonlinear layers (CAP > 2) and Schmidhuber considers CAP > 10 to be very deep learning

A chain of transformations from input to output is a Credit Assignment Path or CAP. For a feedforward neural network, the depth of the CAPs, and thus the depth of the network, is the number of hidden layers plus one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.