10
$\begingroup$

I was reading this book about deep learning by Ian and Aron. In the description of DBN they says DBN has fallen out of favor and is rarely used.

Deep belief networks demonstrated that deep architectures can be successful, by outperforming kernelized support vector machines on the MNIST dataset ( Hinton et al. , 2006 ). Today, deep belief networks have mostly fallen out of favor and are rarely used, even compared to other unsupervised or generative learning algorithms, but they are still deservedly recognized for their important role in deep learning history.

I don't understand why.

Improve this question
$\endgroup$
1
  • 3
    $\begingroup$ This seems like a highly subjective question. My first (completely informal) response would be that the authors wrote this passage a few years back and forgot to update it... The Google Trends graph shows a clearly peak around 2004-2005 but it seems to obviously gain some stream in the last couple of years (2014 onwards). Deep learning, a superset of DBN is "all-the-rage" right now I would say. (Even our server infrastructure engineer at work asked me the other day if we do "deep mining"...) $\endgroup$
    – usεr11852
    Feb 13, 2017 at 23:07

1 Answer 1

Reset to default
4
$\begingroup$

Remember that backpropagation used to come with one big problem; the vanishing gradient; I think the main reason for what deep belief networks are rarely used is because backpropagation used with RELU (Rectified Linear Unit) solves the vanishing gradient problem and it is not an issue anymore and you don´t need to implement a DBN.

The second reason is because even though you could resolve the same problem using similar approaches, big deep networks architectures become way more complex to train with deep belief networks. Using backpropagation with RELU you can train in one shot.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.