1
$\begingroup$

Related Question: Learning weights in a Boltzmann Machine

I'm trying to understand RBMs and how they are applied in training of Deep Architecture. Being new to the field of statistics, I stumbled upon various notations.

Question 1: What does this symbol represent: <$s_is_j$>

It is said while updating weights of RBMs that, the binary states of each node in hidden unit is set to 1 with a probability p equal to a function (a sigmoid function with the input: the sum of product of visible vector and corresponding weights vector connecting a hidden node).

Question 2: Why is every hidden unit being set to 1 (with probability p) and what is the function of the probability p?

Question 3: As far as I understand weights are updated once the system goes to equilibrium, so when is the system said to be in equilibrium?

Also is there any step-by-step procedures for training a network using RBMs

$\endgroup$
2
$\begingroup$

There is a paper, An Introduction to Restricted Boltzmann Machines, that may help you.

For the first question,if you can understand the 9th and 30th formulas in this paper, you will get the answer.

"Why is every hidden unit being set to 1 ", I think there is no sense if hidden($H_j$) unit set $0$, because no matter what the $V_i$ and $W_{ij}$ set to be (assumption $V_i$ connected $H_j$ by parameter $W_{ij}$), the result is $V_i*W_{ij}*H_j=0$ (in the formula $E(V,H)$ used in energy model); so we hope it to be $1$.

It's not important to know the time when is the system said to be in equilibrium, equilibrium occurs when $P(H(i)|V(i))= P(H(i+L)|V(i+L))$.

$\endgroup$
0
$\begingroup$

I really love Edwin Chen's article on RBMs, and I think it will help you understand the algorithm. His code is extremely readable.

I've also ported this code to R, if you prefer that language.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.