I'm trying to understand the difference between a restricted Boltzmann machine (RBM), and a feed-forward neural network (NN). I know that an RBM is a generative model, where the idea is to reconstruct the input, whereas an NN is a discriminative model, where the idea is the predict a label. But what I am unclear about, is why you cannot just use a NN for a generative model? In particular, I am thinking about deep belief networks and multi-layer perceptrons.
Suppose my input to the NN is a set of notes called
x, and my output of the NN is a set of nodes
y. In a discriminative model, my loss during training would be the difference between
y, and the value of
y that I want
x to produce (e.g. probabilities for class labels). However, what about if I just made the output have the same number of nodes as the input, and then set the loss to be the difference between
y? In this way, the network would learn to reconstruct the input, like in an RBM.
So, given that a NN (or a multi-layer perceptron) can be used to train a generative model, why would you use an RBM (or a deep belief network) instead? Or in this case, would they be exactly the same?