Questions tagged [deep-belief-networks]

A type of deep neural network architecture that allows layer-wise unsupervised pre-training.

Filter by
Sorted by
Tagged with
2
votes
0answers
257 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
2
votes
1answer
480 views

Why is backpropagation used more for fine-tuning than the up-down algorithm for deep belief networks?

Deep belief networks are pre-trained using RBMs then fine tuned for a supervised learning task. For almost every paper that I have read, I have seen back-propagation used instead of the up-down ...
2
votes
1answer
1k views

When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised fine-...
2
votes
1answer
84 views

Using Gibbs Sampling on Deep Belief Network with PCA [closed]

I'll make this question as clear as possible: If I were to PCA my data onto say 300 Principal components. Then train a deep belief network with 300 input features. Would I still be able to sample ...
2
votes
0answers
275 views

Using deep learning for time series prediction with uncertain time series window size!

I'm new in area of deep learning and I am trying to use deep learning to do prediction on machine generated log data gathered as stream of data. I have seen LSTM an how it can be helpful to train ...
8
votes
1answer
495 views

Deep learning algorithm

What's the difference between deep belief network and deep convex network?
3
votes
1answer
370 views

Do deep belief networks minimize required domain expertise, pre-preprocessing, and selection of features?

I'm trying to get a basic layman's grasp of deep belief networks and deep learning in general. I've read a few papers and watched a few presentations, but there's one aspect I'm hoping someone can ...
2
votes
1answer
2k views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
2
votes
0answers
303 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
1
vote
0answers
219 views

What is “Hierarchical Probabilistic Inference” in Honglak Lee's C-DBN?

This question is based on Honglak Lee's paper "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations". I have implemented a convolutional RBM with ...
7
votes
3answers
3k views

How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as : $$\ln(L(\theta|v))=\ln(p(v|\theta))=\ln\frac{1}{Z}\sum_h e^{-E(v,h)}=\ln\sum_h e^{E(v,h)}-...
5
votes
2answers
12k views

In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc. And train the ...
8
votes
2answers
5k views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. Inputs to the autoencode() function include lambda, beta, rho and epsilon. What are the bounds for these ...
1
vote
1answer
675 views

definition of deep belief network

I was studying Deep Belief Network (DBN) and have questions. 1) According to the definition of DBN, DBN is formed by stacking RBM on top of each other such that the hidden layer in a lower layer ...
38
votes
3answers
30k views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
5
votes
2answers
1k views

Comparing different deep learning models?

Does anyone know a paper that describes the differences and compares the different deep learning architectures? like Stacked autoencoders, deep believe networks, maxout networks ... etc.
28
votes
2answers
13k views

Deep belief networks or Deep Boltzmann Machines?

I'm confused. Is there a difference between Deep belief networks and Deep Boltzmann Machines? If so, what's the difference?
4
votes
2answers
2k views

Is a Gaussian-Gaussian RBM just a linear model?

The 'conventional' configuration of RBMs are Binary-Binary and Gaussian-Binary (and sometimes Binary-Gaussian) units. Although it is possible for both the visible and hidden units to be gaussian, ...
7
votes
1answer
2k views

Are Restricted Boltzmann Machines better than Stacked Auto encoders and why?

So I'm learning about deep learning. I first learned about stacked auto-encoders and now I'm learning about Restricted Boltzmann Machines. However non in the papers/tutorials I read I found them ...
1
vote
0answers
131 views

Rough estimates for training time of deep belief networks

I'm still learning about deep learning. However I'm currently interested to know if deep learning architectures scale well or not. Suppose I have a dataset with 1 million training examples, can you ...
3
votes
1answer
905 views

Why features compression is good?

I'm reading about deep learning and that in principles it's a features compression technique and that is why it works. Now my question is why compressing features from 200 or so into 4 is better? How ...
1
vote
0answers
129 views

Deep Belief Networks: connecting visible bias of higher layers to hidden bias of lower layer?

Suppose we are building a DBN (Deep Belief Network) and we have already trained some lower layers as Restricted Bolzmann Machines. Now we add a new layer, with new weights and new biases for the new ...
5
votes
1answer
505 views

Can a deep belief network (stacked RBMS) be used solely as a dataset generator?

I have a large dataset (tens of thousands of predictors) on which I would like to perform feature reduction with the intent of better model-building for prediction. Deep Belief Networks seem to ...
1
vote
2answers
2k views

How to update weights in RBM (Restricted Boltzmann Machines)?

Related Question: Learning weights in a Boltzmann Machine I'm trying to understand RBMs and how they are applied in training of Deep Architecture. Being new to the field of statistics, I stumbled ...
20
votes
2answers
2k views

Where and why does deep learning shine?

With all the media talk and hype about deep learning these days, I read some elementary stuff about it. I just found that it is just another machine learning method to learn patterns from data. But my ...
3
votes
1answer
1k views

Deep Belief Network (number of layers)

So we have "several RBMs" Deep Belief Network ...
54
votes
5answers
40k views

Using deep learning for time series prediction

I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to ...
2
votes
1answer
657 views

Sampling from a Deep Belief Network: Treatment of biases in directed part of the model

When generating samples from a DBN, how do you handle the biases that have been learned for the layers below? I know that you normally perform a number of Block Gibbs sampling steps in the undirected ...
11
votes
3answers
2k views

Mathematically modeling neural networks as graphical models

I am struggling to make the mathematical connection between a neural network and a graphical model. In graphical models the idea is simple: the probability distribution factorizes according to the ...
63
votes
3answers
48k views

What is the difference between a neural network and a deep belief network?

I am getting the impression that when people are referring to a 'deep belief' network that this is basically a neural network but very large. Is this correct or does a deep belief network also imply ...
7
votes
2answers
4k views

Deep belief network performs worse than a simple MLP

I tried to train a deep belief network to recognize digits from the MNIST dataset. Everything works OK, I can train even quite a large network. The problem is that the best DBN is worse than a simple ...
1
vote
2answers
452 views

Can deep belief networks be applied to sparse feature vectors/classification problems?

I am trying to beat the performance of an SVM classifier in a text classification task. Input is a bag of words model of sentences with 1 representing presence and 0 representing absence. Output is 1 ...
56
votes
8answers
46k views

R libraries for deep learning

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, neuralnet,...
11
votes
2answers
2k views

How to understand a convolutional deep belief network for audio classification?

In "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations" by Lee et. al.(PDF) Convolutional DBN's are proposed. Also the method is evaluated for image ...