# Questions tagged [restricted-boltzmann-machine]

a Restricted Boltzmann Machine (RBM) is a kind of artificial neural network.

143 questions
Filter by
Sorted by
Tagged with
4 views

### Energy function of a RBM

My question is simply why the Energy function of a RBM is given by: $$E = -\sum_{i,j} w_{ij} \, v_i \, h_j -\sum_i \alpha_i \, v_i - \sum_i \beta_i \, h_i$$ when the energy of the Boltzmann Machine is ...
23 views

### Notation for calculating $p(v)$ marginalising over $h$ in an Restricted Boltzmann Machine

I am working through equation 22 in Introduction to Boltzmann Machines I am a little confused with the notation, in particular in the line: As I understand it, we want the probability of a specific ...
20 views

### Boltzmann machine method of evolution

When a Boltzmann machine modeling binary data 'evolves' to a lower energy state, does it typically evolve via the hebbian update rule (like hopfield nets), or gradient descent? My understanding is ...
53 views

14 views

### Detailed mathematical derivation of the energy function and joint probability of restricted Boltzmann machine

I am wondering how the joint distribution of restricted Boltzmann machine (RBM) is mathematically derived. Consider a RBM with input layer $\bf x$ (with bias $\mathbf b$), hidden layer $\bf h$ (with ...
1 vote
32 views

### Using -1 instead of 0 in binary Restricted Boltzmann Machines

I am aware of the fact that binary Restricted Boltzmann Machines - which use $0$ or $1$ as states - are asymmetric, in the sense that two states have to be both $1$ for $w_{ij}$ to enter in the energy ...
58 views

### Training in Restricted boltzman machine

I am having doubt in training part of RBM's. I am confused between whats the difference between training RBM by block gibbs sampling and training RBM using contrastive divergence?
1 vote
154 views

### Deep Belief Network

I am a bit confused about deep belief networks. Should the RBM output be the input to the feed forward neural network for the fine tuning step or just the weights of the neural network have to be ...
302 views

### Is this overfitting?

The Use Case: We are given three unique, 'ground truth' binary training patterns (not patterns with 'noise'). A machine is to be trained with these three vectors. The requirement is that once trained, ...
78 views

### Sampling from deep belief networks

DBNs are generative models, and usually you sample by thermalising the deepest layer (as it's a restricted Boltzmann Machine), and then forward propagating a sample towards the visible layer to get a ...
117 views

### What is difference between attention and restricted-Boltzmann machine (RBM)?

Attention has attracted lots of interest recently. However, when I looked at the details of calculation, I recognize a lot of similarity between RBM and attention. Self-attention (key=value) is ...
65 views

### Subsequent data transformation in Deep Belief Networks

I have a Deep Belief Network made of 4 Restricted Boltzmann Machines. The lowest level RBM seems to train correctly. The deeper layers however seem not to learn anything at all. I plot the receptive ...
75 views

### Computing Gradients for a [-1, 1]-valued RBM

The gradient derivation for a binary-valued RBM with values $\in\{0,1\}$ is well-documented, for example in Goodfellow, et al and here on Cross Validated. However, in some works (e.g., associative ...
86 views

### Proving RBM similar to feedforward neural network with single latent layer

In an examination, I came across a question where I was asked to prove that RBMs are similar to a feedforward neural network with a single layer. I have an intuition that the structure is similar (...
215 views

### What does "stochastic" mean in RBM networks?

I am learning about restricted Boltzmann machines (RBM). It is still unclear to me why it is a stochastic algorithm. I was wondering if some one can explain it with comparison to deterministic ones (...
1k views

### Is initializing the weights of autoencoders still a difficult problem?

I was wondering if initializing the weights of autoencoders is still difficult and what the most recent strategies are for it. I have been reading different articles. In one of Hinton's papers (2006)...
1 vote
977 views

### Energy function of Restricted Boltzmann Machine (RBM)

The energy function for RBM (Restricted Boltzmann Machine) is defined as $$E(v,h) = -\sum_{i,j} w_{ij} \, v_i \, h_j -\sum_i a_i \, v_i - \sum_i b_i \, h_i$$ with the joint distribution  \tag{1} p(...
1 vote
228 views

### Deep learning book RBM Equition 20.15 how is the conditional distribution derived?

I was reading the DL textbook from here. I understand how the probability of individual elements of hidden layer h gets derived. However, I could not understand the equation (20.15), which derive the ...
1 vote
324 views

### Feature selection using Restricted Boltzmann Machine

I am new in the field of RBMs, DBMs and I cannot understand some things. I came across the idea of feature selection using RBMs (or Deep Belief Networks). Although the Hidden nodes which make new ...
155 views

### Can I use Free Energy as reconstruction error when I use RBM to anomaly detection?

Can I use Free Energy as reconstruction error when I use RBM to anomaly detection? If Free Energy of a sample more a threshold, can I regard it an outlier? How to explain Free Energy of RBM?
147 views

### Why a Deep Belief Network has connection that points to the input layer?

Supposing to have a 3-layer DBN. I don't understand the specific reason for which the connections between the top two layers are undirected and the connections between all other layers are directed. ...
23 views

### Randomize dataset for Restricted Boltmann Machines

Suppose I want to train a RBM (or even a DBN architecture) and then fine-tune the parameter training a Feedforward NN. In my case the dataset is composed of time series, so in principle there is a ...
83 views

### RBM to predict multiple output continuous response

0 I came across a paper where they are predicting GPA/Grades of different courses for a candidate using RBM. My question is can we apply RBM for a problem where the output needs to be continuous and ...
1 vote
422 views

### Feature extraction vs Fine tuning with Restricted Boltmann Machines

I am reading a paper which uses a Restricted Boltzmann Machine to extract features from a dataset in an unsupervised way and then use those features to train a classifier (they use SVM but it could be ...
72 views

### Why can a Restricted Boltzmann Machine reconstruct using same weights?

Commonly NN (autoencoders) use a set of weights in the reduction process and another in the reconstruction process. But a RBM uses the same weights in construction and reconstruction process. I don't ...
1 vote
284 views

### Stacked RBM vs. Bernoulli RBM

I'm requested to implement mnist project using stacked RBM. I didn't find any implementation of RBM in keras or tensorflow. However, there is a Bernoulli RBM in sklearn. Could you please guide me if ...
1k views

### Why can't we use backpropagation and gradient descent on a Restricted Boltzmann Machine

Can someone please explain why we cannot use the backpropagation algorithm and gradient descent to train a Restricted Boltzmann Machine. In other words, why can't we train an RBM in the same manner ...
120 views

### Can someone explain in layman terms how the Contrastive Divergence algorithm works step by step?

I am interested in learning about Restricted Boltzmann Machines (RBMs), but I have trouble understanding how an RBM is trained using contrastive divergence. There are only few papers on this topic and ...
324 views

### Summary of Pre-Training a Neural Network with Stacks of RBMs

I understand that pre-training with stacks of RBMs is now (mostly) obsolete but I'm still interested in knowing if I have the right idea on how it is done. Say you have a basic neural network with a ...
42 views

### Probability of the input features in Boltzmann Machine

On the youtube lecture at 8:15(Restricted Boltzmann machine - free energy) professor mentioned, that we want to make the probability of the observed features to be big. However I didn't understand ...
216 views

### Gibbs Sampling vs. Using Raw Probability in Contrastive Divergence

In Hinton's Practical Guide to Training Restricted Boltzmann Machines, Section 3, he discusses different situations in which one should take a sample from the Gibbs sampling process, and other ...
171 views

1k views

### KL divergence derivation

I want to understand KL divergene. Can someone please explain why we need inequality lnx
185 views

### partition function therm in the derivation of log likelihood in RBM?

In the following derivation when we take derivative of second term why the summation is only over h although it should be be over v and h because we are dealing with Z.
348 views

### Conditional probability expression in restricted boltzmann machine? How to get p(h=1|v)?

I'm reading a book on Deep learning and having trouble in deriving an expression Can someone please explain how to go from equation(20.10) to (20.11)? How summation term is converting into product ...
163 views

### How does a hidden unit in a Boltzmann Machine differ from a hidden unit in a Neural Network?

I was reading about Boltzmann Machines, and I found this: The state of a hidden unit in a Boltzmann Machine is a random variable, but in a Neural Net it is a deterministic function of the inputs. ...
1 vote
1k views

### Why is the log likelihood used for the loss function in an RBM

In an RBM, the loss function is defined like this: Why are we using the log likelihood function? How does that measure the error?