Deep belief networks are pre-trained using RBMs then fine tuned for a supervised learning task. For almost every paper that I have read, I have seen back-propagation used instead of the up-down algorithm. Why? The up-down algorithm is a generative training approach and allows us to sample from the network.


1 Answer 1


It is really a choice based on your particular use-case and what performance you get with the methods you try.

Back-prop is well understood in terms of implementation and is also easier to implement as compared to the up-down method.

If you have a really deep network then the vanishing gradient problem can leave the hidden layers closer to the input layer untouched in terms of the supervised learning, but if the network is not that deep (you can measure the gradient at each layer to see how bad the drop off is) then you can find the back-prop training reaches all the hidden layers.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.