The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
10 views

How to add extra layer of MLP to DBN

I am trying to add MLP layer to DBN that can use final parameters of DBN model as Input for MLP model. I am new to python so am not well versed with its input and output processes. Any help is ...
0
votes
0answers
6 views

Rea-time or batch unsupervised techique & tool for a high-dimensional binary data

Its a theoretical question, no real data just yet to access. Hence, I cannot tell the application. Data have thousands of features an billion instances. In addition, every instance has a unique ...
0
votes
0answers
11 views

What is negative reconstruction error for Deep Belief Network?

I was studying Deep Belief Networks and started testing a small example. For the sake of these question consider the one at deeplearning tutorials Observation 1: I noticed that my reconstruction ...
2
votes
1answer
52 views

Why are Hinton's multilayer deep-learning networks stochastic?

First I'll sum up my intuitive (beginner) understanding of his deep-learning architecture. A short summary can be listened to on Coursera in the 5 minute video. We start with several layers of ...
1
vote
0answers
48 views

How is free energy an unnormalized conditional log-probability?

I am following Bengio's Learning Deep Architectures for AI and at page 28 there is a phrase that confuses me: $a(x)$ is the discriminant function or an unnormalized conditional log-probability, ...
0
votes
0answers
35 views

Why is my DBN predict only 2 out of 5 classes?

I'm using the Deeplearning.net DBN tutorial to train my data set. I normalize the feature set to zero-mean-unit-variance. However, I can only get the network to predict 2 out 5 classes even though the ...
0
votes
0answers
39 views

Difference in training procedure for DBN and DBM

This is related to the following thread Deep belief networks or Deep Boltzmann Machines? but it doesn't seem to answer in a practical sense what the difference is. So I gather a DBN is directed and ...
1
vote
0answers
46 views

Deep Learning with few features available

I was asked to employ deep learning on some seismic simulation data. Visually, the data is a cube, 1000 x 1000 x 1000. For each point in the cube, there are 3 numeric features [1, 0]. Some of it is ...
1
vote
0answers
96 views

Updating bias with RBMs (Restricted Boltzmann Machines)

Am very new to RBMs, trying to write an RBM program now. Sorry if this is a silly question and/or answered on here already. I've read a few articles online, and questions on here, but I can't find ...
3
votes
1answer
116 views

Need pointers to deep learning tutorials [closed]

I'm looking for good study material about deep belief networks, with particular emphasis to classification and feature extraction tasks for non-image data. I don't seem to find a great deal about ...
2
votes
1answer
1k views

What is the architecture of a Stacked Convolutional Autoencoder

So I am trying to do pre training on images of humans using convolutional nets. I read the papers http://people.idsia.ch/~ciresan/data/icann2011.pdf and ...
1
vote
0answers
142 views

Threshold on tanh or sigmoid in Convolutional neural network

I have read several papers on Convolutional Neural Nets but I am yet to come across any that has used thresholds on tanh or sigmoid to decide whether the neuron will fire or not. Obviously this works ...
0
votes
1answer
241 views

How to normalize filters in convolutional neural networks?

Usually when convolving images the elements in the filter sum to one. Does this creteria enforce in convolutional neural networks? If yes, How?
1
vote
0answers
65 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
0
votes
0answers
69 views

Why is backpropagation used more for fine-tuning than the up-down algorithm for deep belief networks?

Deep belief networks are pre-trained using RBMs then fine tuned for a supervised learning task. For almost every paper that I have read, I have seen back-propagation used instead of the up-down ...
2
votes
1answer
196 views

When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised ...
0
votes
0answers
32 views

Does a Restricted Boltzmann Machine model any distribution as a Gibbs distribution?

What i know is that a Restricted Boltzmann Machine (RBM) is a Markov Random Field (MRF) and that the joint distribution of an MRF represents a Gibbs distribution. What I also know is that our goal of ...
2
votes
1answer
42 views

Using Gibbs Sampling on Deep Belief Network with PCA [closed]

I'll make this question as clear as possible: If I were to PCA my data onto say 300 Principal components. Then train a deep belief network with 300 input features. Would I still be able to sample ...
1
vote
0answers
92 views

Using deep learning for time series prediction with uncertain time series window size!

I'm new in area of deep learning and I am trying to use deep learning to do prediction on machine generated log data gathered as stream of data. I have seen LSTM an how it can be helpful to train ...
6
votes
1answer
276 views

Deep learning algorithm

What's the difference between deep belief network and deep convex network?
2
votes
1answer
221 views

number of feature maps in convolutional neural networks

When learning convolutional neural network, I have questions regarding the following figure. 1) C1 in the layer 1 has 6 feature maps, does that mean there are six convolutional kernals? Each ...
0
votes
0answers
24 views

Training of Joint Layer in Deep Belief Networks?

I am reading a paper "A deep learning approach to Machine Transliteration". In this paper they speak about using Deep Belief Nets for Machine Transliteration. I have a basic understanding of RBMs( ...
2
votes
1answer
161 views

Do deep belief networks minimize required domain expertise, pre-preprocessing, and selection of features?

I'm trying to get a basic layman's grasp of deep belief networks and deep learning in general. I've read a few papers and watched a few presentations, but there's one aspect I'm hoping someone can ...
2
votes
1answer
71 views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
0
votes
0answers
136 views

How to scale data to train RBMs?

I know that when training Bernoulli Restricted Boltzmann Machines with real-valued data, then the input data should be scaled to the interval $[0,1]$ (last section from here). What I understood is ...
1
vote
0answers
289 views

Where can I find a MATLAB implementation of Convolutional Deep Belief Network?

I have been trying to find a MATLAB implementation of the Convolutional Deep Belief Network. A Google search returned libraries that implement a Convolutional Restricted Boltzmann machine. I am aware ...
1
vote
0answers
141 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
0
votes
0answers
98 views

How do Convolutional deep belief networks stack with a restricted Boltzman machine?

In a conventional DBN, the activation probabilities of one RBM are used as the input of the next one. Therefore, the number of hidden units of the first RBM is the number of visible units in the ...
1
vote
0answers
85 views

What is “Hierarchical Probabilistic Inference” in Honglak Lee's C-DBN?

This question is based on Honglak Lee's paper "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations". I have implemented a convolutional RBM with ...
12
votes
4answers
10k views

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between convolutional neural networks (CNN), restricted Boltzmann ...
1
vote
2answers
148 views

How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as : $$\ln(L(\theta|v))=\ln(p(v|\theta))=\ln\frac{1}{Z}\sum_h e^{-E(v,h)}=\ln\sum_h ...
0
votes
2answers
1k views

In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc. And train the ...
3
votes
2answers
984 views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...
1
vote
0answers
54 views

Is deep belief network a belief network [closed]

I am new to deep learning. Please help me.. What is belief network. What is the use of it? How to learn belief networks? Is deep belief network a belief network with multiple layers?
3
votes
1answer
264 views

Summarization of text documents (legal domain) using deep learning techniques

I am referring to the site deeplearning.net on how to implement the deep learning architectures. I have read quite a few research papers on document summarization (both single document and ...
0
votes
1answer
159 views

definition of deep belief network

I was studying Deep Belief Network (DBN) and have questions. 1) According to the definition of DBN, DBN is formed by stacking RBM on top of each other such that the hidden layer in a lower layer ...
4
votes
3answers
844 views

Guideline to select the hyperparameters in Deep Learning

I'm looking for a paper that could help in giving a guideline on how to choose the hyperparameters of a deep architecture, like stacked auto-encoders or deep believe networks. There are a lot of ...
0
votes
1answer
133 views

Comparing different deep learning models?

Does anyone know a paper that describes the differences and compares the different deep learning architectures? like Stacked autoencoders, deep believe networks, maxout networks ... etc.
11
votes
1answer
1k views

Deep belief networks or Deep Boltzmann Machines?

I'm kinda confused. Is there a difference between Deep belief networks and Deep Boltzmann Machines? Are they different stuff or same thing?! If so, what's the difference?
4
votes
2answers
387 views

Is a Gaussian-Gaussian RBM just a linear model?

The 'conventional' configuration of RBMs are Binary-Binary and Gaussian-Binary (and sometimes Binary-Gaussian) units. Although it is possible for both the visible and hidden units to be gaussian, ...
3
votes
1answer
592 views

Are Restricted Boltzmann Machines better than Stacked Auto encoders and why?

So I'm learning about deep learning. I first learned about stacked auto-encoders and now I'm learning about Restricted Boltzmann Machines. However non in the papers/tutorials I read I found them ...
1
vote
0answers
64 views

Rough estimates for training time of deep belief networks

I'm still learning about deep learning. However I'm currently interested to know if deep learning architectures scale well or not. Suppose I have a dataset with 1 million training examples, can you ...
1
vote
1answer
238 views

Why features compression is good?

I'm reading about deep learning and that in principles it's a features compression technique and that is why it works. Now my question is why compressing features from 200 or so into 4 is better? How ...
1
vote
0answers
62 views

Deep Belief Networks: connecting visible bias of higher layers to hidden bias of lower layer?

Suppose we are building a DBN (Deep Belief Network) and we have already trained some lower layers as Restricted Bolzmann Machines. Now we add a new layer, with new weights and new biases for the new ...
4
votes
1answer
226 views

Can a deep belief network (stacked RBMS) be used solely as a dataset generator?

I have a large dataset (tens of thousands of predictors) on which I would like to perform feature reduction with the intent of better model-building for prediction. Deep Belief Networks seem to ...
0
votes
2answers
369 views

How to update weights in RBM (Restricted Boltzmann Machines)?

Related Question: Learning weights in a Boltzmann Machine I'm trying to understand RBMs and how they are applied in training of Deep Architecture. Being new to the field of statistics, I stumbled ...
11
votes
2answers
1k views

Where and why does deep learning shine?

With all the media talk and hype about deep learning these days, I read some elementary stuff about it. I just found that it is just another machine learning method to learn patterns from data. But my ...
9
votes
3answers
1k views

How true is this slide on deep learning?

I was listening to a talk and saw this slide: How true is it?
2
votes
1answer
309 views

Deep Belief Network (number of layers)

So we have "several RBMs" Deep Belief Network ...
28
votes
5answers
12k views

Using deep learning for time series prediction

I'm new in area of deep learning and for me first step was to read interesting articles from deeplearning.net site. In papers about deep learning, Hinton and others mostly talk about applying it to ...