Good question: note that in the field of Deep Learning things are not always as well-cut and clearly defined as in Statistical Learning (also because there's a lot of hype), so don't expect to find definitions as rigorous as in Mathematics. Anyway, the multilayer perceptron is a specific feed-forward neural network architecture, where you stack up multiple fully-connected layers (so, no convolution layers at all), where the activation functions of the hidden units are often a sigmoid or a tanh. The nodes of the output layer usually have softmax activation functions (for classification) or linear activation functions (for regression). The typical MLP architectures are not "deep", i.e., we don't have many hidden layers. You usually have, say, 1 to 5 hidden layers. These neural networks were common in the '80, and are trained by backpropagation.
Now, with Deep Neural Network we mean a network which has many layers (19, 22, 152,...even > 1200, though that admittedly is very extreme). Note that
- we haven't specified the architecture of the network, so this could be feed-forward, recurrent, etc.
- we haven't specified the nature of the connections, so we could have fully connected layers, convolutional layers, recurrence, etc.
- "many" layers admittedly is not a rigorous definition.
So, why does it still make sense to speak of DNNs (apart from hype reasons)? Because when you start stacking more and more layers, you actually need to use new techniques (new activation functions, new kind of layers, new optimization strategies...even new hardware) to be able to 1) train your model and 2) make it generalize on new cases. For example, suppose you take a classical MLP for 10-class classification, tanh activation functions, input & hidden layers with 32 units each and output layer with 10 softmax units $\Rightarrow 32\times32+32\times10 = 1344$ weights. You add 10 layers $\Rightarrow 11584$ weights. This is a minuscule NN by today's standards. However, when you go on to train it on a suitably large data set, you find that the convergence rate has slowed down tremendously. This is not only due to the larger number of weights, but to the vanishing gradient problem - back-propagation computes the gradient of the loss function by multiplying errors across each layers, and these small numbers become exponentially smaller the more layers you add. Thus, the errors don't propagate (or propagate very slowly) down your network, and it looks like the error on the training set stops decreasing with training epochs.
And this was a small network - the deep Convolutional Neural Networks called AlexNet had 5 layers but 60 millions weights, and it's considered small by today's standards! When you have so many weights, then any data set is "small" - even ImageNet, a data set of images used for classification, has "only" about 1 million images, thus the risk of overfitting is much larger than for shallow network.
Deep Learning can thus be understood as the set of tools which are used in practice to train neural networks with a large number of layers and weights, achieving low generalization error. This task poses more challenges than for smaller networks. You can definitely build a Deep Multilayer Perceptron and train it - but (apart from the fact that it's not the optimal architecture for many tasks where Deep Learning is used today) you will probably use tools which are different from those used when networks used to be "shallow". For example, you may prefer ReLU activation units to sigmoid or tanh, because they soften the vanishing gradient problem.
