0
$\begingroup$

I've been reading through some stuff related to Restricted Boltzmann Machines (RBM). By wikipedia for example I can see the graphical model is in general very similar to a standard Feed Forward Network (FFN), but on the other hand there's the training which is based on Gibbs Sampling.

What I don't get though is the difference, if there's any, between RBM and FFN, apart from the training process. The basic algorithm to train a ANN is gradient descent, but you can use in general other optimization methods, such as L-BFGS for example.

So if the difference is only the training method it doesn't seem to me they're different networks.

On the other hand some people when introduce RBM starts with probabilistic graphical models, which makes me think maybe an RBM is different from a FFN.

Can you clarify what exactly is an RBM then?

$\endgroup$
2
$\begingroup$

A feed forward neural network is just a function $f(x; \theta)$ of some inputs and its weights. There is a clear procedure for computing the values of each unit/layer in the network from the previous layer. You can ascribe your own interpretation for what the outputs mean (a probability value, an RL policy, a point estimate, etc) and train the model accordingly.

On the other hand, an RBM models $P(x; \theta)$. It doesn't have any outputs and can't do anything other than tell you the probability of the data given the parameters. You can't compute the value of any unobserved variables in the RBM, because they're random variables.

$\endgroup$
  • $\begingroup$ So what's the difference with a bayesian network then? $\endgroup$ – user8469759 Apr 13 at 1:00
  • $\begingroup$ @user8469759 a bayes net is a directed graphical model, whereas RBMs are undirected $\endgroup$ – shimao Apr 13 at 1:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.