The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
15 views

Activations or Binary States for RBM Input

I understand that the activations or the sampled binary states can be used as input for an RBM or for the inputs of subsequent layers when pre-training a DBN - from the MILA DBN tutorial; Hintons 06 ...
0
votes
0answers
24 views

Visualising features learnt by deep auto encoders

I am trying to visualise the weights learnt by autoencoders. The task is straight-forward when only one hidden layer is used. Say the image is 10*10, so both the input and output layers will have 100 ...
3
votes
1answer
38 views

Why is training a deep convolutional neural network taking longer time than anticipated?

It's taking me over 4 days to train a deep learning network with just 10000 images of 224px x 224px x 3 channels size, with batch size 25. The machine has 32GB RAM, a Core i7 CPU, and a GTX 960 GPU. I'...
6
votes
0answers
59 views

Restricted Boltzmann Machine : how is it used in machine learning?

Background: Yes, Restricted Boltzmann Machine (RBM) CAN be used to initiate the weights of a neural network. Also it CAN be used in a "layer-by-layer" way to build a deep belief network (that is, to ...
1
vote
0answers
28 views

Deep Belief Network configuration for dice face recognition

I should develop a network that can read the result of throwing a dice. I have a dataset which consists on a synthetic collection of such images, together with the corresponding target values. Each ...
0
votes
0answers
19 views

Why use Tied Weight in Deep Belief Network?

I am reading about the Deep Belief Network from here. As per my understanding, DBN can be thought of multiple layers of RBM stacked on each other, and in order to train a DBN, we need to train each ...
1
vote
1answer
64 views

Output of Restricted Boltzmann Machine energy function

I'm just starting to learn about Restricted Boltzmann Machines (RBMs). So far I understand that they try to predict/output the inputs that are fed to them. If the RBM is trained on input where each ...
0
votes
0answers
15 views

Want unsupervised deep hierarchical representation of images : DBM or DBN?

If we were performing classification, then the fine-tuning can be achieved via backprop after the initial unsupervised greedy layer-wise training. But in case of just representation learning where ...
0
votes
1answer
12 views

In the Restricted Boltzmann Machine's free energy function, how can it be simplified to only be a function of v?

I realize this is a copy of this question: How to compute the free energy of a RBM given its energy? however I am unable to comment on the best answer to ask that person to elaborate. We start with: ...
1
vote
2answers
255 views

How to derive this conditional distribution function for a Restricted Boltzmann Machine?

I am following along Ian Goodfellow's new Deep Learning book and, reading the last chapter, I am confused about equations 20.7-20.9. We have a joint distribution function, $P(v,h)$, and we are ...
0
votes
0answers
43 views

What if I'm getting always around the same error on Training, Validation and Test set?

currently playing around with H20 and R using a Feed-Forward Neural Net. I'm doing parameter space exploration - than means I'm trying to tweak the various parameters of the NN in a set of nested ...
2
votes
2answers
162 views

What makes neural network a *convolutional* neural network?

What is the difference between a Convolutional Neural Network (CNN) and an ordinary Neural Network (NN)? What does convolution mean in this context?
0
votes
0answers
42 views

Online Machine Learning of sequential events with varying delay

Lets say we have A to Z features which repeat sequentially. So you have A(1), B(1), ... Z(1) at time 1 followed by A(2), B(2),....Z(2) at time 2 and so on till A(n), B(n), ... Z(n) at time n. Each of ...
3
votes
1answer
65 views

how do you handle a “none of these” class in a CNN

It is the closed-world assumption of a CNN. For example I have trained a CNN to recognize, sedans, jeeps, trucks, suvs and crossovers, and I present an airplane it tries to fit it into of these 5. How ...
1
vote
1answer
110 views

how to handle small datasets with large dimensions

I have 48 samples which are case and control and 27000 features for each sample so my matrix is [48 X 27000]and I am using Deep belief networks(DBN) as my algorithm to predict the accuracy of the ...
0
votes
0answers
72 views

Use deep belief networks for unsupervised anomaly detection

I am working on anomaly detection on data with a large number of variables (>50) with continuous values. As I have read that deep belief networks can be used for unsupervised anomaly detection on ...
1
vote
0answers
52 views

Greedy Training of Deep Belief Networks

I try to understand the justification of Greedy Training for Deep Belief Networks. I read the tutorial at http://deeplearning.net/tutorial/DBN.html and various papers of Hinton,Bengio and other ...
1
vote
2answers
291 views

Dimensionality reduction: RBM autoencoders vs. de-noising autoencoders

I am looking at non-linear dimensionality reduction techniques and am currently trying to understand the practical differences between different autoencoder approaches: Can somebody point me to a ...
5
votes
2answers
176 views

The bottleneck of applying deep learning in practice

After reading a lot of deep learning papers, a kind of rough feeling is that there exist a lot of tricks in training the network to get the better-than-normal performance. From an industry application ...
1
vote
1answer
216 views

What is the best Deep Learning Library in R? [closed]

I am looking for a complete deep learning library in R. I am trying to find one or more libraries to implement: Recurrent NN Deep Belief NN Convolutional NN I have tried multiple libraries such as ...
0
votes
0answers
128 views

normalization of input to a deep belief network

I am playing around with deep belief networks, and am unsure what the best normalization scheme is. (Note that by this I mean a deep network that is trained in a greedy layer-wise manner, by ...
0
votes
0answers
57 views

What is relation between RBM and DBN? [duplicate]

What is relation between Restricted Boltzmann Machine, Deep Belief Networks, Deep Boltzmann Machines? Are they related to deep learning or to graphical models(Probabilistic Graphical Models)? ...
0
votes
1answer
46 views

How to add extra layer of MLP to DBN

I am trying to add MLP layer to DBN that can use final parameters of DBN model as Input for MLP model. I am new to python so am not well versed with its input and output processes. Any help is welcome....
0
votes
0answers
10 views

Rea-time or batch unsupervised techique & tool for a high-dimensional binary data

Its a theoretical question, no real data just yet to access. Hence, I cannot tell the application. Data have thousands of features an billion instances. In addition, every instance has a unique ...
0
votes
0answers
37 views

What is negative reconstruction error for Deep Belief Network?

I was studying Deep Belief Networks and started testing a small example. For the sake of these question consider the one at deeplearning tutorials Observation 1: I noticed that my reconstruction ...
2
votes
1answer
106 views

Why are Hinton's multilayer deep-learning networks stochastic?

First I'll sum up my intuitive (beginner) understanding of his deep-learning architecture. A short summary can be listened to on Coursera in the 5 minute video. We start with several layers of ...
1
vote
0answers
90 views

How is free energy an unnormalized conditional log-probability?

I am following Bengio's Learning Deep Architectures for AI and at page 28 there is a phrase that confuses me: $a(x)$ is the discriminant function or an unnormalized conditional log-probability, ...
0
votes
0answers
90 views

Why is my DBN predict only 2 out of 5 classes?

I'm using the Deeplearning.net DBN tutorial to train my data set. I normalize the feature set to zero-mean-unit-variance. However, I can only get the network to predict 2 out 5 classes even though the ...
2
votes
0answers
183 views

Deep Learning with few features available

I was asked to employ deep learning on some seismic simulation data. Visually, the data is a cube, 1000 x 1000 x 1000. For each point in the cube, there are 3 numeric features [1, 0]. Some of it is ...
6
votes
1answer
515 views

Updating bias with RBMs (Restricted Boltzmann Machines)

Am very new to RBMs, trying to write an RBM program now. Sorry if this is a silly question and/or answered on here already. I've read a few articles online, and questions on here, but I can't find ...
3
votes
1answer
140 views

Need pointers to deep learning tutorials [closed]

I'm looking for good study material about deep belief networks, with particular emphasis to classification and feature extraction tasks for non-image data. I don't seem to find a great deal about ...
12
votes
2answers
5k views

What is the architecture of a Stacked Convolutional Autoencoder

So I am trying to do pre training on images of humans using convolutional nets. I read the papers http://people.idsia.ch/~ciresan/data/icann2011.pdf and http://ai.stanford.edu/~ang/papers/nips10-...
1
vote
0answers
389 views

Threshold on tanh or sigmoid in Convolutional neural network

I have read several papers on Convolutional Neural Nets but I am yet to come across any that has used thresholds on tanh or sigmoid to decide whether the neuron will fire or not. Obviously this works ...
1
vote
2answers
630 views

How to normalize filters in convolutional neural networks?

Usually, when convolving images, the elements in the filter sum to one. Is this criterion enforced in convolutional neural networks? If yes, how?
1
vote
1answer
118 views

A paper that proves using the latent features of RBM as input to logistic regression?

I'm looking for a paper that includes a proof that simply training a Restricted Boltzmann Machine and then using the latent features as input to a logistic regression classifier is a correct thing and ...
1
vote
0answers
166 views

Why is backpropagation used more for fine-tuning than the up-down algorithm for deep belief networks?

Deep belief networks are pre-trained using RBMs then fine tuned for a supervised learning task. For almost every paper that I have read, I have seen back-propagation used instead of the up-down ...
2
votes
1answer
439 views

When should we use Gibbs Sampling in a deep belief network? Before or after fine-tuning?

Gibbs sampling allows for sampling a vector with a deep belief network. There are two steps to training a DBN for a supervised learning task: greedy unsupervised pre-training and supervised fine-...
2
votes
1answer
57 views

Using Gibbs Sampling on Deep Belief Network with PCA [closed]

I'll make this question as clear as possible: If I were to PCA my data onto say 300 Principal components. Then train a deep belief network with 300 input features. Would I still be able to sample ...
1
vote
0answers
162 views

Using deep learning for time series prediction with uncertain time series window size!

I'm new in area of deep learning and I am trying to use deep learning to do prediction on machine generated log data gathered as stream of data. I have seen LSTM an how it can be helpful to train ...
7
votes
1answer
393 views

Deep learning algorithm

What's the difference between deep belief network and deep convex network?
4
votes
1answer
890 views

number of feature maps in convolutional neural networks

When learning convolutional neural network, I have questions regarding the following figure. 1) C1 in the layer 1 has 6 feature maps, does that mean there are six convolutional kernals? Each ...
2
votes
1answer
284 views

Do deep belief networks minimize required domain expertise, pre-preprocessing, and selection of features?

I'm trying to get a basic layman's grasp of deep belief networks and deep learning in general. I've read a few papers and watched a few presentations, but there's one aspect I'm hoping someone can ...
2
votes
1answer
309 views

Scale invariance for images

Given that images can be of vastly different resolutions, but neural networks are usually presented as having a fixed number of inputs, what are the standard techniques used to handle the difference ...
1
vote
0answers
496 views

Where can I find a MATLAB implementation of Convolutional Deep Belief Network?

I have been trying to find a MATLAB implementation of the Convolutional Deep Belief Network. A Google search returned libraries that implement a Convolutional Restricted Boltzmann machine. I am aware ...
1
vote
0answers
200 views

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced good results?

What are some of the image classification datasets other than MNIST on which Deep Belief Network (DBN) has produced state-of-the-art results? Even if its not state-of-the-art, but, I am looking for ...
1
vote
0answers
151 views

What is “Hierarchical Probabilistic Inference” in Honglak Lee's C-DBN?

This question is based on Honglak Lee's paper "Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations". I have implemented a convolutional RBM with ...
42
votes
4answers
36k views

What is the difference between convolutional neural networks, restricted Boltzmann machines, and auto-encoders?

Recently I have been reading about deep learning and I am confused about the terms (or say technologies). What is the difference between convolutional neural networks (CNN), restricted Boltzmann ...
2
votes
3answers
397 views

How to derive the gradient formula for the Maximum Likelihood in RBM?

I am learning RBM (restricted Boltzmann machine) for deep learning. The log-likelihood of RBM is given as : $$\ln(L(\theta|v))=\ln(p(v|\theta))=\ln\frac{1}{Z}\sum_h e^{-E(v,h)}=\ln\sum_h e^{E(v,h)}-...
0
votes
2answers
4k views

In convolutional neural networks, how to prevent the overfitting?

Given certain amount of labeled data, we define the net structure, such as number of layers, types of layers, the number of convolutional layers, the number of pooling layers, etc. And train the ...
4
votes
2answers
2k views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...