The tag has no wiki summary.

learn more… | top users | synonyms

5
votes
1answer
127 views

Deep learning algorithm

What's the difference between deep belief network and deep convex network?
1
vote
0answers
14 views

number of feature maps in convolutional neural networks

When learning convolutional neural network, I have questions regarding the following figure. 1) C1 in the layer 1 has 6 feature maps, does that mean there are six convolutional kernals? Each ...
0
votes
0answers
13 views

Training of Joint Layer in Deep Belief Networks?

I am reading a paper "A deep learning approach to Machine Transliteration". In this paper they speak about using Deep Belief Nets for Machine Transliteration. I have a basic understanding of RBMs( ...
2
votes
1answer
32 views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
0
votes
0answers
37 views

How to scale data to train RBMs?

I know that when training Bernoulli Restricted Boltzmann Machines with real-valued data, then the input data should be scaled to the interval $[0,1]$ (last section from here). What I understood is ...
1
vote
0answers
105 views

Where can I find a MATLAB implementation of Convolutional Deep Belief Network?

I have been trying to find a MATLAB implementation of the Convolutional Deep Belief Network. A Google search returned libraries that implement a Convolutional Restricted Boltzmann machine. I am aware ...
0
votes
0answers
52 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
0
votes
0answers
44 views

How do Convolutional deep belief networks stack with a restricted Boltzman machine?

In a conventional DBN, the activation probabilities of one RBM are used as the input of the next one. Therefore, the number of hidden units of the first RBM is the number of visible units in the ...
0
votes
0answers
27 views

What is “Hierarchical Probabilistic Inference” in Honglak Lee's C-DBN?

This question is based on Honglak Lee's paper "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations". I have implemented a convolutional RBM with ...
3
votes
4answers
775 views

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between convolutional neural networks (CNN), restricted Boltzmann ...
0
votes
0answers
46 views

How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as : and its gradient w.r.t. the parameter is: I don't understand how is the gradient ...
0
votes
0answers
32 views

3 Conceptual Questions about a DBN (Deep Belief Network)

Paper is from here: https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf (1) Why is there a RBM sitting on top? (2) Why is a RBM considered equivalent to an infinite sigmoid network? (3) Is a chain ...
0
votes
1answer
227 views

In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc. And train the ...
0
votes
0answers
30 views

GB-RBM unable to learn and generate simple 2D-Motions?

I am trying to apply a GB-RBM to a variation of the LASA Handwriting Dataset to be able to generate new examples. My dataset contains motion trajectories of simple shapes like 'S' shapes, spirals, 'C' ...
0
votes
0answers
190 views

How to implement a Gaussian-Rectified RBM and generate data?

I have a bit trouble implementing a gaussian-rectified RBM. I would like to show you how I would implement it. It would be nice if you point out errors or comment the implementation. These are the ...
0
votes
0answers
30 views

Associate different continuous types of data with Restricted Boltzmann Machines

This demo shows the DBNs capability to associate different input modalities ("images of digits" and "labels"). By clamping one modality at the top layer, the network can infer the other via (Gibbs) ...
3
votes
1answer
349 views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...
0
votes
0answers
36 views

Time and space complexity of Deep Belief Nets (DBN)

What is the time and space(memory) complexity of DBNs? given d:number of dimensions(attributes), n:number of records, and l:number of hidden layers.
1
vote
0answers
36 views

Is deep belief network a belief network [closed]

I am new to deep learning. Please help me.. What is belief network. What is the use of it? How to learn belief networks? Is deep belief network a belief network with multiple layers?
3
votes
1answer
134 views

Summarization of text documents (legal domain) using deep learning techniques

I am referring to the site deeplearning.net on how to implement the deep learning architectures. I have read quite a few research papers on document summarization (both single document and ...
0
votes
1answer
75 views

definition of deep belief network

I was studying Deep Belief Network (DBN) and have questions. 1) According to the definition of DBN, DBN is formed by stacking RBM on top of each other such that the hidden layer in a lower layer ...
3
votes
2answers
194 views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
0
votes
1answer
75 views

Comparing different deep learning models?

Does anyone know a paper that describes the differences and compares the different deep learning architectures? like Stacked autoencoders, deep believe networks, maxout networks ... etc.
5
votes
1answer
435 views

Deep belief networks or Deep Boltzmann Machines?

I'm kinda confused. Is there a difference between Deep belief networks and Deep Boltzmann Machines? Are they different stuff or same thing?! If so, what's the difference?
0
votes
0answers
24 views

Deep belief nets and PCA

I have been using successfully DBNs for classification for MNIST and some other tasks. I was thinking of doing dimensionality reduction before training, in order to boost performance. However, when I ...
0
votes
0answers
164 views

Is a Gaussian-Gaussian RBM just a linear model?

The 'conventional' configuration of RBMs are Binary-Binary and Gaussian-Binary (and sometimes Binary-Gaussian) units. Although it is possible for both the visible and hidden units to be gaussian, ...
3
votes
1answer
324 views

Are Restricted Boltzmann Machines better than Stacked Auto encoders and why?

So I'm learning about deep learning. I first learned about stacked auto-encoders and now I'm learning about Restricted Boltzmann Machines. However non in the papers/tutorials I read I found them ...
1
vote
0answers
48 views

Rough estimates for training time of deep belief networks

I'm still learning about deep learning. However I'm currently interested to know if deep learning architectures scale well or not. Suppose I have a dataset with 1 million training examples, can you ...
1
vote
1answer
111 views

Why features compression is good?

I'm reading about deep learning and that in principles it's a features compression technique and that is why it works. Now my question is why compressing features from 200 or so into 4 is better? How ...
1
vote
0answers
33 views

Deep Belief Networks: connecting visible bias of higher layers to hidden bias of lower layer?

Suppose we are building a DBN (Deep Belief Network) and we have already trained some lower layers as Restricted Bolzmann Machines. Now we add a new layer, with new weights and new biases for the new ...
0
votes
0answers
71 views

Can not classify Iris dataset by continuous deep belief network

We have a case to map real input values into low dimensional binary vectors. So we use CDBN here, but whatever I change the parameter combinations of k, lr or epochs, it always failed to classify our ...
3
votes
1answer
155 views

Can a deep belief network (stacked RBMS) be used solely as a dataset generator?

I have a large dataset (tens of thousands of predictors) on which I would like to perform feature reduction with the intent of better model-building for prediction. Deep Belief Networks seem to ...
0
votes
2answers
165 views

How to update weights in RBM (Restricted Boltzmann Machines)?

Related Question: Learning weights in a Boltzmann Machine I'm trying to understand RBMs and how they are applied in training of Deep Architecture. Being new to the field of statistics, I stumbled ...
10
votes
2answers
524 views

Where and why does deep learning shine?

With all the media talk and hype about deep learning these days, I read some elementary stuff about it. I just found that it is just another machine learning method to learn patterns from data. But my ...
6
votes
3answers
576 views

How true is this slide on deep learning?

I was listening to a talk and saw this slide: How true is it?
2
votes
1answer
195 views

Deep Belief Network (number of layers)

So we have "several RBMs" Deep Belief Network ...
21
votes
3answers
6k views

Using deep learning for time series prediction

I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to ...
1
vote
1answer
166 views

Sampling from a Deep Belief Network: Treatment of biases in directed part of the model

When generating samples from a DBN, how do you handle the biases that have been learned for the layers below? I know that you normally perform a number of Block Gibbs sampling steps in the undirected ...
1
vote
1answer
197 views

Mathematically modeling neural networks as graphical models

I am struggling to make the mathematical connection between a neural network and a graphical model. In graphical models the idea is simple: the probability distribution factorizes according to the ...
16
votes
2answers
5k views

What is the difference between a neural network and a deep belief network?

I am getting the impression that when people are referring to a 'deep belief' network that this is basically a neural network but very large. Is this correct or does a deep belief network also imply ...
4
votes
2answers
1k views

Deep belief network performs worse than a simple MLP

I tried to train a deep belief network to recognize digits from the MNIST dataset. Everything works OK, I can train even quite a large network. The problem is that the best DBN is worse than a simple ...
1
vote
2answers
242 views

Can deep belief networks be applied to sparse feature vectors/classification problems?

I am trying to beat the performance of an SVM classifier in a text classification task. Input is a bag of words model of sentences with 1 representing presence and 0 representing absence. Output is 1 ...
26
votes
6answers
9k views

R libraries for deep learning

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, ...