Questions tagged [deep-belief-networks]

A type of deep neural network architecture that allows layer-wise unsupervised pre-training.

Filter by
Sorted by
Tagged with
0
votes
0answers
20 views

Does Neural Network or Deep Learning consider correlation among variables? [duplicate]

If there are any correlations among variables, do I need to think of choosing least correlated variables for neural network or deep learning? I am just wondering because Regression Analysis care about ...
0
votes
1answer
29 views

Feature selection using Restricted Boltzmann Machine

I am new in the field of RBMs, DBMs and I cannot understand some things. I came across the idea of feature selection using RBMs (or Deep Belief Networks). Although the Hidden nodes which make new ...
0
votes
1answer
19 views

Why a Deep Belief Network has connection that points to the input layer?

Supposing to have a 3-layer DBN. I don't understand the specific reason for which the connections between the top two layers are undirected and the connections between all other layers are directed. ...
1
vote
0answers
83 views

How to calculate associated degree of freedom of linked nodes in graph

I am working on text analytics and building a knowledge graph with high frequency entities (noun chunks) as graph nodes and their linkage between co-occurrence in a sentence as edges. I am able to ...
3
votes
1answer
91 views

What is an “undirected associative memory” in Hinton et al 2006?

In A fast learning algorithm for deep belief nets, the authors use the term "undirected associative memory". I am not sure what they are referring to, and unfortunately Google searches for this term ...
2
votes
0answers
36 views

Choosing the number of hidden layers and nodes in a Deep Belief Network

What are the recent advances and current best practices in choosing the number and size of stacked Restricted Boltzmann Machines in Deep Belief Networks ?
1
vote
0answers
154 views

Is this a really a belief propagation problem?

BACKGROUND This is basically a reputation problem that involves a set of interacting entities $e_i$. Each entity has, in principle, a reputation vector $\vec{b}_i$. That reputation depends on what ...
3
votes
0answers
69 views

State of the Art Status of Deep Boltzmann Machine and Pretraining

I have been reading some old papers by Hinton on deep Boltzmann machine and deep belief networks, but I wonder what the current status is regarding these models: Are DBM and DBN totally outdated? I ...
0
votes
1answer
1k views

Deep belief networks: supervised or unsupervised?

I want to know whether a Deep Belief Network (or DBN) is a supervised learning algorithm or an unsupervised learning algorithm? After lot of research into DBN working I am confused at this very ...
2
votes
0answers
101 views

Do stacked RBM's have any benefits/advantages over CNN?

Do stacked RBM's have any benefits/advantages over CNN? If the concern is about face recognition.
-1
votes
1answer
236 views

2 questions about the functions in the `deepnet` Deep Neural Network package in R

I'm using R to perform the Deep Neural Network. But there are so many packages and functions related to neural networks that I am confused. I am wondering about the following two things. What are ...
2
votes
0answers
233 views

Neural Network converges to a constant [duplicate]

I'm having a similar problem to the following post (Feed-Forward) Neural Networks keep converging to mean. The model is built with Deep Neural Network library in Matlab by Masayuki Tanaka. The ...
0
votes
1answer
53 views

Need advice regarding Deep Learning for predictive model

I have been working with neural networks for generation of a predictive model using a multivariate approach. I have come across Deep Learning (or Deep neural networks) as a tool to enhance the success ...
1
vote
0answers
203 views

Understanding Deep Belief Networks!

I have implemented Stacked Autoencoder in tensorflow and I was thinking of implementing Deep Belief Networks using Stacked RBM's. I had started reading about DBN's from various websites and through ...
1
vote
1answer
360 views

Deep Learning for sequences

I want to use deep learning techniques to perform better inference tasks than Hidden Markov Models (which is a shallow model)? I was wondering what is the state-of-the art deep learning model to ...
1
vote
0answers
233 views

Clustering of time series using RBMs/DBN?

I have a sequence of actions dataset. There are 10 different actions, but lets say for simplicity that I have a1 and a2 actions. The data are not stationary. For some time we have one distribution of ...
2
votes
0answers
277 views

What are the most important parameters to tune in a deep belief network?

I am trying to create a Deep Belief Network (DBF) for a binary classification problem. The nolearn package provides a good library for implementing them. I see that there are very many parameters to ...
7
votes
1answer
2k views

Why are deep belief networks (DBN) rarely used?

I was reading this book about deep learning by Ian and Aron. In the description of DBN they says DBN has fallen out of favor and is rarely used. Deep belief networks demonstrated that deep ...
2
votes
1answer
1k views

Deep Learning: What happens after each epoch?

I am trying to understand batch size and epochs, and I found this very helpful. Each epoch is all of the data, lets say 10,000 rows, and the number of batches is the number of groups the epoch is ...
3
votes
2answers
501 views

Why does pre-training help avoid the vanishing gradient problem?

I read that a problem with the Classic approach to deep NN is the vanishing gradient, which is caused by the derivative of the logistic activation function - broadly speaking, the update flowing down ...
2
votes
1answer
157 views

Boltzmann machines: learning algorithm

I'm trying to study Boltzmann machines, so I don't undestand this recurrent formulation for the training stage of the weights $w$: $\Delta w_{ij} = E_{data} (v_i h_j ) − E_{model} (v_i h_j )$ all ...
2
votes
1answer
2k views

The learning rule for sigmoid belief nets

On page 12 of this tutorial, it shows that the learning rule is $\Delta w_{ji} = \epsilon s_j(s_i-p_i)$. Can someone show me how this is derived? I got $(1-p_i)s_j$ instead after computing $\frac{d\...
8
votes
3answers
7k views

Feature selection using deep learning?

I want to calculate the importance of each input feature using deep model. But I found only one paper about feature selection using deep learning - deep feature selection. They insert a layer of ...
2
votes
0answers
120 views

boltzmann machine; from logistic function to boltzmann distribution

I'm trying to understand BM; on this topic, tutorials explain it with two formulas: logistic function for the probabilty of single units $p(unit=1)=\frac{1}{1+e^{-\sum\limits_xwx}}$ and, when the ...
0
votes
1answer
27 views

Is there any papers or blogs that discuss the effect of embedding layer dimensionality?

Is there any paper or blog that discuss the effect of the dimensionality of embedding layers? The Embedding layers can be used in deep learning models, like CNN or LSTM.
10
votes
4answers
11k views

difference between neural network and deep learning

In terms of the difference between neural network and deep learning, we can list several items, such as more layers are included, massive data set, powerful computer hardware to make training ...
3
votes
1answer
559 views

Do I have to normalize the input vector for RNN if it only consists of 0 and 1?

Yan LeCun's paper "Efficient Backprop" indicates that the average of each input variable over training set should be close to zero. If the input variables are all categorical variables and encoded ...
8
votes
1answer
4k views

Is Greedy Layer-Wise Training of Deep Networks necessary for successfully training or is stochastic gradient descent enough?

Is it possible to achieve state of the art results by using back-propagation only (without pre-training) ? Or is it so that all record breaking approaches use some form of pre-training ? Is back-...
2
votes
0answers
168 views

Deep Belief Net applied to Netflix Prize?

In Restricted Boltzmann Machines for Collaborative Filtering Restricted Boltzmann Machines (RBMs) are applied to the Netflix Prize data set. An obvious next step might be to use stacks of RBMs (i.e. ...
2
votes
1answer
2k views

Why is training a deep convolutional neural network taking longer time than anticipated?

It's taking me over 4 days to train a deep learning network with just 10000 images of 224px x 224px x 3 channels size, with batch size 25. The machine has 32GB RAM, a Core i7 CPU, and a GTX 960 GPU. I'...
21
votes
2answers
2k views

Restricted Boltzmann Machine : how is it used in machine learning?

Background: Yes, Restricted Boltzmann Machine (RBM) CAN be used to initiate the weights of a neural network. Also it CAN be used in a "layer-by-layer" way to build a deep belief network (that is, to ...
1
vote
0answers
181 views

Deep Belief Network configuration for dice face recognition [duplicate]

I should develop a network that can read the result of throwing a dice. I have a dataset which consists on a synthetic collection of such images, together with the corresponding target values. Each ...
1
vote
1answer
414 views

In the Restricted Boltzmann Machine's free energy function, how can it be simplified to only be a function of v?

I realize this is a copy of this question: How to compute the free energy of a RBM given its energy? however I am unable to comment on the best answer to ask that person to elaborate. We start with: ...
2
votes
2answers
659 views

How to derive this conditional distribution function for a Restricted Boltzmann Machine?

I am following along Ian Goodfellow's new Deep Learning book and, reading the last chapter, I am confused about equations 20.7-20.9. We have a joint distribution function, $P(v,h)$, and we are ...
5
votes
1answer
1k views

how do you handle a “none of these” class in a CNN

It is the closed-world assumption of a CNN. For example I have trained a CNN to recognize, sedans, jeeps, trucks, suvs and crossovers, and I present an airplane it tries to fit it into of these 5. How ...
2
votes
1answer
2k views

how to handle small datasets with large dimensions

I have 48 samples which are case and control and 27000 features for each sample so my matrix is [48 X 27000]and I am using Deep belief networks(DBN) as my algorithm to predict the accuracy of the ...
1
vote
0answers
292 views

Greedy Training of Deep Belief Networks

I try to understand the justification of Greedy Training for Deep Belief Networks. I read the tutorial at http://deeplearning.net/tutorial/DBN.html and various papers of Hinton,Bengio and other ...
2
votes
2answers
1k views

Dimensionality reduction: RBM autoencoders vs. de-noising autoencoders

I am looking at non-linear dimensionality reduction techniques and am currently trying to understand the practical differences between different autoencoder approaches: Can somebody point me to a ...
9
votes
2answers
1k views

The bottleneck of applying deep learning in practice

After reading a lot of deep learning papers, a kind of rough feeling is that there exist a lot of tricks in training the network to get the better-than-normal performance. From an industry application ...
1
vote
1answer
451 views

What is the best Deep Learning Library in R? [closed]

I am looking for a complete deep learning library in R. I am trying to find one or more libraries to implement: Recurrent NN Deep Belief NN Convolutional NN I have tried multiple libraries such as ...
0
votes
1answer
140 views

How to add extra layer of MLP to DBN

I am trying to add MLP layer to DBN that can use final parameters of DBN model as Input for MLP model. I am new to python so am not well versed with its input and output processes. Any help is welcome....
2
votes
1answer
148 views

Why are Hinton's multilayer deep-learning networks stochastic?

First I'll sum up my intuitive (beginner) understanding of his deep-learning architecture. A short summary can be listened to on Coursera in the 5 minute video. We start with several layers of ...
1
vote
0answers
285 views

How is free energy an unnormalized conditional log-probability?

I am following Bengio's Learning Deep Architectures for AI and at page 28 there is a phrase that confuses me: $a(x)$ is the discriminant function or an unnormalized conditional log-probability, ...
0
votes
0answers
193 views

Why is my DBN predict only 2 out of 5 classes? [duplicate]

I'm using the Deeplearning.net DBN tutorial to train my data set. I normalize the feature set to zero-mean-unit-variance. However, I can only get the network to predict 2 out 5 classes even though the ...
2
votes
0answers
328 views

Deep Learning with few features available

I was asked to employ deep learning on some seismic simulation data. Visually, the data is a cube, 1000 x 1000 x 1000. For each point in the cube, there are 3 numeric features [1, 0]. Some of it is ...
8
votes
1answer
2k views

Updating bias with RBMs (Restricted Boltzmann Machines)

Am very new to RBMs, trying to write an RBM program now. Sorry if this is a silly question and/or answered on here already. I've read a few articles online, and questions on here, but I can't find ...
3
votes
1answer
159 views

Need pointers to deep learning tutorials [closed]

I'm looking for good study material about deep belief networks, with particular emphasis to classification and feature extraction tasks for non-image data. I don't seem to find a great deal about ...
23
votes
3answers
10k views

What is the architecture of a stacked convolutional autoencoder?

So I am trying to do pretraining on images of humans using convolutional nets. I read the papers (Paper1 and Paper2) and this stackoverflow link, but I am not sure I am understand the structure of the ...
2
votes
0answers
1k views

Threshold on tanh or sigmoid in Convolutional neural network

I have read several papers on Convolutional Neural Nets but I am yet to come across any that has used thresholds on tanh or sigmoid to decide whether the neuron will fire or not. Obviously this works ...
1
vote
2answers
2k views

How to normalize filters in convolutional neural networks?

Usually, when convolving images, the elements in the filter sum to one. Is this criterion enforced in convolutional neural networks? If yes, how?