Questions tagged [deep-belief-networks]

A type of deep neural network architecture that allows layer-wise unsupervised pre-training.

Filter by
Sorted by
Tagged with
37
votes
3answers
29k views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
56
votes
8answers
46k views

R libraries for deep learning

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, neuralnet,...
62
votes
3answers
47k views

What is the difference between a neural network and a deep belief network?

I am getting the impression that when people are referring to a 'deep belief' network that this is basically a neural network but very large. Is this correct or does a deep belief network also imply ...
11
votes
3answers
2k views

Mathematically modeling neural networks as graphical models

I am struggling to make the mathematical connection between a neural network and a graphical model. In graphical models the idea is simple: the probability distribution factorizes according to the ...
7
votes
1answer
2k views

Are Restricted Boltzmann Machines better than Stacked Auto encoders and why?

So I'm learning about deep learning. I first learned about stacked auto-encoders and now I'm learning about Restricted Boltzmann Machines. However non in the papers/tutorials I read I found them ...
7
votes
2answers
4k views

Deep belief network performs worse than a simple MLP

I tried to train a deep belief network to recognize digits from the MNIST dataset. Everything works OK, I can train even quite a large network. The problem is that the best DBN is worse than a simple ...
7
votes
1answer
2k views

Why are deep belief networks (DBN) rarely used?

I was reading this book about deep learning by Ian and Aron. In the description of DBN they says DBN has fallen out of favor and is rarely used. Deep belief networks demonstrated that deep ...
7
votes
3answers
3k views

How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as : $$\ln(L(\theta|v))=\ln(p(v|\theta))=\ln\frac{1}{Z}\sum_h e^{-E(v,h)}=\ln\sum_h e^{E(v,h)}-...
1
vote
2answers
2k views

How to normalize filters in convolutional neural networks?

Usually, when convolving images, the elements in the filter sum to one. Is this criterion enforced in convolutional neural networks? If yes, how?