10
$\begingroup$

In terms of the difference between neural network and deep learning, we can list several items, such as more layers are included, massive data set, powerful computer hardware to make training complicated model possible.

Besides these, are there any more detailed explanation regarding the difference between NN and DL?

$\endgroup$
  • 3
    $\begingroup$ As far as I know, just having several hidden layers is sufficient to make a network "deep;" more data and larger computers are more a symptom of the increasing availability of both for machine learning tasks. $\endgroup$ – Sycorax Sep 12 '16 at 18:39
  • $\begingroup$ Perhaps this question should be migrated to the new artifical intelligence stack exchange? $\endgroup$ – WilliamKF Sep 12 '16 at 22:22
  • 2
    $\begingroup$ @WilliamKF This is squarely on-topic here. $\endgroup$ – Sycorax Sep 13 '16 at 17:42
7
$\begingroup$

Deep learning = deep artificial neural networks + other kind of deep models.

Deep artificial neural networks = artificial neural networks with more than 1 layer. (see minimum number of layers in a deep neural network)

$\endgroup$
3
$\begingroup$

Frank Dernoncourt has a better general purpose answer, but I think it's worth mentioning that when people use the broad term "Deep Learning" they're often implying the use of recent techniques, like convolution, that you wouldn't find in older/traditional (fully-connected) neural networks. For image recognition problems, convolution can enable deeper neural networks because convoluted neurons/filters reduce the risk of overfitting somewhat by sharing weights.

$\endgroup$
1
$\begingroup$

Neural networks with a lot of layers are deep architectures.

However, the backpropagation learning algorithm used in neural networks doesn't work well when the network is very deep. Learning architectures in deep architectures ("deep learning") have to address this. For example, Boltzmann machines use a contrastive learning algorithm instead.

Coming up with a deep architecture is easy. Coming up with a learning algorithm that works well for a deep architecture has proven difficult.

$\endgroup$
  • $\begingroup$ But it seems that backpropagation algorithm is still used to train conv net and recurrent net, even though they leverage some newly developed numerical optimization techniques, such as batch normalization. $\endgroup$ – user3269 Sep 12 '16 at 21:30
  • $\begingroup$ @user3269 batch normalization and dropout are examples of modifications to the learning algorithm to try to make them work well in deep architectures. $\endgroup$ – Neil G Sep 12 '16 at 22:04
0
$\begingroup$

Deep learning requires a neural network having multiple layers —  each layer doing mathematical transformations and feeding into the next layer. The output from the last layer is the decision of the network for a given input. The layers between the input and output layer are called hidden layers.

A deep learning neural network is a massive collection of perceptrons interconnected in layers. The weights and bias of each perceptron in the network influence the nature of the output decision of the entire network. In a perfectly tuned neural network, all the values of weights and bias of all the perceptron are such that the output decision is always correct (as expected) for all possible inputs. How are the weights and bias configured? This happens iteratively during the training of the network —  called deep learning. (Sharad Gandhi)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.