Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rbm]

Restricted Boltzman Machine

1
vote
1answer
12 views

Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
0
votes
1answer
16 views

Why can a Restricted Boltzmann Machine reconstruct using same weights?

Commonly NN (autoencoders) use a set of weights in the reduction process and another in the reconstruction process. But a RBM uses the same weights in construction and reconstruction process. I don't ...
0
votes
0answers
9 views

Why is it most energetically favourable for feature extraction to occur in energy-based models like RBMs?

My current understanding of Restricted Boltzmann Machines (RBMs) is as follows. Please correct me if I'm wrong, as misunderstanding RBMs may be the cause of my question. An RBM is an energy-based ...
0
votes
0answers
12 views

Is it possible to build autoencoders based on stack of RBMs?

If I have real-valued data, is it possible implement an autoencoder using stacked RBMs where the input and output layers have real-valued data while all layers in between have binary RBMs?
0
votes
0answers
14 views

Wrong weights learned when training RBM

I'm training my RBM network and on epoch #4 I have such a filters representation (my weights matrix) But on the next iteration (fifth epoch) something went wrong and my filters became like this What ...
0
votes
1answer
84 views

Stacked RBM vs. Bernoulli RBM

I'm requested to implement mnist project using stacked RBM. I didn't find any implementation of RBM in keras or tensorflow. However, there is a Bernoulli RBM in sklearn. Could you please guide me if ...
1
vote
1answer
238 views

Why can't we use backpropagation and gradient descent on a Restricted Boltzmann Machine

Can someone please explain why we cannot use the backpropagation algorithm and gradient descent to train a Restricted Boltzmann Machine. In other words, why can't we train an RBM in the same manner ...
3
votes
0answers
87 views

Can someone explain in layman terms how the Contrastive Divergence algorithm works step by step?

I am interested in learning about Restricted Boltzmann Machines (RBMs), but I have trouble understanding how an RBM is trained using contrastive divergence. There are only few papers on this topic and ...
0
votes
1answer
95 views

Summary of Pre-Training a Neural Network with Stacks of RBMs

I understand that pre-training with stacks of RBMs is now (mostly) obsolete but I'm still interested in knowing if I have the right idea on how it is done. Say you have a basic neural network with a ...
0
votes
1answer
33 views

Probability of the input features in Boltzmann Machine

On the youtube lecture at 8:15(Restricted Boltzmann machine - free energy) professor mentioned, that we want to make the probability of the observed features to be big. However I didn't understand ...
0
votes
0answers
113 views

Training RBM with (normal) contrastive divergence vs persistent contrastive divergence

I implemented RBM by using PyTorch here and trained it with CalTech 101 28x28 Silhouettes dataset containing binary images. I implemented both (naive) contrastive divergence and persistent ...
3
votes
1answer
71 views

Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
0
votes
1answer
54 views

Isn't computing the “tractable error” in Restricted Boltzmann Machines (RBM) intractable?

Let $v \in \{0,1\}^M$ be the visible layer, $h \in \{0,1\}^N$ be the hidden layer, where $M$ and $N$ are natural numbers. Given the biases $b \in \Re^M$, $c \in \Re^N$ and weights $W \in \Re^{M \times ...
1
vote
1answer
548 views

Restricted Boltzmann Machines vs GAN

Can someone please tell me how RBMs and GANs compare to each other? I know that the community is more excited over GANs than RBMs at the moment. I guess it's because GANs produce better results? My ...
0
votes
1answer
80 views

explanation of RBM definition

I'm learning about RBM and I try to understand the notation used for it. We have the input vector $v=(v_i)$ and the output vector $h=(h_j)$, a weight matrix $W=(W_{ij})$ and finally two bias vectors - ...
2
votes
1answer
337 views

Why do we want (Restricted) Boltzmann Machines to be stochastic?

Boltzmann machines are stochastic recurrent neural networks with a specific architecture, and they are interesting in several contexts. They are stochastic in the sense that the sampling procedure ...
2
votes
0answers
95 views

Do stacked RBM's have any benefits/advantages over CNN?

Do stacked RBM's have any benefits/advantages over CNN? If the concern is about face recognition.
6
votes
1answer
201 views

Why is MCMC not reliable when compared to stochastic gradient descent?

I came across the following quote on enter link description here if we made MCMC as reliable as stochastic gradient descent now is for deep networks, that could mean a resurgence of more explicit ...
4
votes
2answers
393 views

Restricted Boltzmann Machines - Understanding contrastive divergence vs. ML learning

I've read many articles about RBMs as well as Hinton's original paper about the CD algorithm (at least roughly), but I still don't get the big picture. I understand CD conceptually and I've already ...
0
votes
1answer
409 views

scikit's RBM pseudo likelihood calculation

According to scikit-learn's documentation given here, the pseudo likelihood is calculated by; Computing the free energy on X, then on a randomly corrupted version of X, and returns the log of the ...
1
vote
2answers
553 views

What does it mean to sample from an RBM?

In restricted Boltzmann machines we ofter say that given $v$ we sample $h$, and also that given $h$ we sample $v$. Can someone explain the physical meaning of this, considering a MNIST image as ...
-1
votes
1answer
464 views

KL divergence derivation

I want to understand KL divergene. Can someone please explain why we need inequality lnx
0
votes
0answers
92 views

partition function therm in the derivation of log likelihood in RBM?

In the following derivation when we take derivative of second term why the summation is only over h although it should be be over v and h because we are dealing with Z.
0
votes
1answer
155 views

Conditional probability expression in restricted boltzmann machine? How to get p(h=1|v)?

I'm reading a book on Deep learning and having trouble in deriving an expression Can someone please explain how to go from equation(20.10) to (20.11)? How summation term is converting into product ...
2
votes
1answer
131 views

How does a hidden unit in a Boltzmann Machine differ from a hidden unit in a Neural Network?

I was reading about Boltzmann Machines, and I found this: The state of a hidden unit in a Boltzmann Machine is a random variable, but in a Neural Net it is a deterministic function of the inputs. ...
0
votes
1answer
394 views

Why is the log likelihood used for the loss function in an RBM

In an RBM, the loss function is defined like this: Why are we using the log likelihood function? How does that measure the error?
2
votes
1answer
326 views

What are fantasy particles in RBMs

In Restricted Boltzman Machines, when collecting the statistics I sometimes heard of fantasy particles being used. What are these? How are they useful?
1
vote
0answers
2k views

Understanding a simple example of restricted Boltzmann machine (RBM)

I am trying to get an intuitive idea of RBMs out of curiosity, and using a simple example on youtube based on preferences for different sports, which denote profiles roughly corresponding to ...
1
vote
1answer
784 views

Restricted Boltzmann Machine: how to implement Gaussian visible units

Please, I'm a newby with Restricted Boltzmann Machine, I'm a psychologist (not very good with math) and and I've some confusion about the use of Gaussian visible units. Note I'm working on the ...
3
votes
0answers
129 views

Visible Layer Bias in Restricted Boltzmann Machines

In Neural Networks the bias term of the hidden units can be considered a threshold for the node to fire. This is how neurons basically work in the brain as well. In RBMs, also a bias for the visible ...
1
vote
0answers
64 views

ReLUs improve Restricted Boltzmann Machines

This question is about the use of Rectified Linear Units as hidden units in Restricted Boltzmann Machines. In Nair and Hinton's paper, using ReLUs as hidden units is proposed. In section 1, they ...
1
vote
0answers
220 views

Clustering of time series using RBMs/DBN?

I have a sequence of actions dataset. There are 10 different actions, but lets say for simplicity that I have a1 and a2 actions. The data are not stationary. For some time we have one distribution of ...
14
votes
3answers
397 views

What does the “machine” in “support vector machine” and “restricted Boltzmann machine” mean?

Why are they called "machines"? Is there an origin to the word "machine" used in this context? (Like the name "linear programming" can be confusing but we know why it is called "programming.")
1
vote
1answer
1k views

Derivation of Restricted Boltzmann Machine Conditional Probability

I was reading Goodfellow, Bengio, and Courville's Deep Learning and in Chapter 20 there is a derivation of the conditional probability $P(h_j=1|\boldsymbol{v})$, the conditional probability of the $j^{...
3
votes
3answers
2k views

Basic confusion about Restricted Boltzmann Machines (RBM)

As I understand it, the standard restricted Boltzmann machine (RBM) exhibits binary stochastic visible and hidden units. The joint probability of the binary and visible units is given by the Boltzmann ...
1
vote
0answers
65 views

question of sampling in a binary-binary RBM

I've learned that the weights of the RBM defines the joint probability distribution of the visible units and hidden units. So I try a very simple matlab code to prove that. I try two methods to ...
2
votes
1answer
149 views

Boltzmann machines: learning algorithm

I'm trying to study Boltzmann machines, so I don't undestand this recurrent formulation for the training stage of the weights $w$: $\Delta w_{ij} = E_{data} (v_i h_j ) − E_{model} (v_i h_j )$ all ...
5
votes
1answer
315 views

P(h|v) conditional probability proof of Restricted Boltzman Machine

I have a question about the last step of the proof of conditional probability of the restricted boltzman machine in the deeplearningbook.org Below are the derivation from the book: Shouldn't ...
3
votes
1answer
2k views

Variational Auto-encoders vs Restricted Boltzmann Machines

What are the differences of modeling ability between Variational Auto-encoders (VAEs) and Restricted Boltzmann Machines (RBMs)? What I am interested in is to know about the unsupervised learning ...
0
votes
1answer
104 views

How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?

I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions. But how do ...
2
votes
1answer
263 views

Sampling from a Convolutional Restricted Boltzmann Machine's Visible Gaussian Real-valued Units

I am trying to confirm whether or not I am understanding the process described in the title. I am implementing a CRMB (with Real Valued Gaussian Visible units and Binary hidden units) as outlined in ...
0
votes
1answer
63 views

Computation differences of RBMs and CNNs

CNNs often surpass RBMs in most computer vision tasks in terms of classification accuracy. However, has there been any analysis comparing the following factors: Hyper-parameter sensitivity Amount of ...
8
votes
3answers
7k views

Feature selection using deep learning?

I want to calculate the importance of each input feature using deep model. But I found only one paper about feature selection using deep learning - deep feature selection. They insert a layer of ...
2
votes
1answer
2k views

LSTM for predicting probabilities

As a newbie to machine learning, I've been playing around with theano and deeplearning4j libraries and as an interesting application I thought of applying long short-term memory (LSTM) to horse racing....
0
votes
2answers
1k views

Getting probability from Restricted Boltzmann Machine

Let's consider a trained Restricted Boltzmann Machine model. It was trained to maximize P(v). Since it's a generative model, how can I get a probability of an input vector which it is supposed to ...
4
votes
0answers
234 views

free energy of variational autoencoder

There is a conception of free energy related to restricted boltzmann machine(RBM). Since variational autoencoder(VAE) is an alternative to RBM for autoencoding, is there a counterpart definition of ...
9
votes
2answers
475 views

Are graphical models and Boltzmann machines related mathematically?

While I have actually done some programming with Boltzmann machines in a physics class, I am not familiar with their theoretical characterization. In contrast, I know a modest amount about the theory ...
2
votes
1answer
212 views

Understanding the distribution of the Spike & Slab Restricted Boltzmann Machine (ssRBM)

The ssRBM is described as a way to model mean and covariance using Restricted Boltzmann Machines. I'm reading the paper that introduced the spike and slab restricted boltzmann machine. I have yet do ...
16
votes
2answers
2k views

Modern Use Cases of Restricted Boltzmann Machines (RBM's)?

Background: A lot of the modern research in the past ~4 years (post alexnet) seems to have moved away from using generative pretraining for neural networks to achieve state of the art classification ...
2
votes
0answers
163 views

Deep Belief Net applied to Netflix Prize?

In Restricted Boltzmann Machines for Collaborative Filtering Restricted Boltzmann Machines (RBMs) are applied to the Netflix Prize data set. An obvious next step might be to use stacks of RBMs (i.e. ...