In terms of the difference between neural network and deep learning, we can list several items, such as more layers are included, massive data set, powerful computer hardware to make training complicated model possible.
Besides these, are there any more detailed explanation regarding the difference between NN and DL?
Deep learning = deep artificial neural networks + other kind of deep models.
Deep artificial neural networks = artificial neural networks with more than 1 layer. (see minimum number of layers in a deep neural network)
Frank Dernoncourt has a better general purpose answer, but I think it's worth mentioning that when people use the broad term "Deep Learning" they're often implying the use of recent techniques, like convolution, that you wouldn't find in older/traditional (fully-connected) neural networks. For image recognition problems, convolution can enable deeper neural networks because convoluted neurons/filters reduce the risk of overfitting somewhat by sharing weights.
Neural networks with a lot of layers are deep architectures.
However, the backpropagation learning algorithm used in neural networks doesn't work well when the network is very deep. Learning architectures in deep architectures ("deep learning") have to address this. For example, Boltzmann machines use a contrastive learning algorithm instead.
Coming up with a deep architecture is easy. Coming up with a learning algorithm that works well for a deep architecture has proven difficult.
Deep learning requires a neural network having multiple layers — each layer doing mathematical transformations and feeding into the next layer. The output from the last layer is the decision of the network for a given input. The layers between the input and output layer are called hidden layers.
A deep learning neural network is a massive collection of perceptrons interconnected in layers. The weights and bias of each perceptron in the network influence the nature of the output decision of the entire network. In a perfectly tuned neural network, all the values of weights and bias of all the perceptron are such that the output decision is always correct (as expected) for all possible inputs. How are the weights and bias configured? This happens iteratively during the training of the network — called deep learning. (Sharad Gandhi)
