# Restricted Boltzmann Machine, simple example (MATLAB preferred)

There are no shortage of RBM literatures, but they are not easy to follow for beginners of RBM. Some example will help a lot. Example either in MATLAB or a specific example with numbers (like this link : http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/)

Note: this example (http://blog.echen.me/2011/07/18/introduction-to-restricted-boltzmann-machines/) is good, but still lack of details of steps.

I ran this code several times, and each time got different result. The last line print(r.run_visible(user)) gives either [0 1] or [0 0] or [1 0] or [1 1]. What does this mean?

Why does RBM not yield consistent results ?

Not sure if you are looking for a clearer introduction to RBM or just an example.

If you prefer to have a look at an actual implementation, I have implemented the RBM in Matlab because I found existing implementations to lack modularity for my own needs. You can find it here: https://github.com/pixelou/nnbox/blob/master/networks/RBM.m (pretrain method starts at line 119). I cannot guaranty that it is bug free though :-)

For a comprehensive introduction to Restricted boltzmann machines, you can have a look at Training restricted Boltzmann machines: An introduction from Asja Fischer & Christian Igel, this is the clearest paper in terms of proofs and structure. But I found it a bit hard to follow on the first read so you can instead split your reading into several stages:

1 - consider the problem of learning a model (RBM) as a minimization of the distance between the parametric pdf of the RBM and the underlying dataset distribution.

2 - compute the gradient of the distance so that you can do gradient descent

I have detailed the computation here: How to derive the gradient formula for the Maximum Likelihood in RBM?

3 - notice that you cannot compute that expression because one part is intractable :-). Use Contrastive divergence (or any other method but CD is the most popular) to approximate this value. You can read A New Learning Algorithm for Mean Field Boltzmann Machines from Welling & Hinton (2001) which is one of the early publications on CD.

4 - Use that gradient to perform gradient update, adjust with regularization and tricks. For that matter, a Practical Guide to Training Restricted Boltzmann Machines from Hinton (2010) is a must read.

• @bill If this answer is correct, please select it, otherwise be more specific in your question. Thanks – pixelou Feb 23 '16 at 15:28
• Hi pixelou Thanks a lot, really appreciate... I am monkeying with your source code, but meet quite some errors/bugs. Could you please provide a version of example that is bug free, i.e., can be run on MATLAB ? // For example, in MNIST_DNN.m line 46, there is "e % LultilayerNet is configured to pretrain layer wise" and apparently "e" is typo. After I deleted this letter "e", ran MNIST_DNN.m again, I got more errors : Error using RBM/vis2hid Too many input arguments. Error in RBM/compute (line 112) Y = self.vis2hid(X, [], false); ....... – Bill Ancalagon the black Feb 25 '16 at 22:47
• Hi bill. Sorry you had to crash-test the example file. I have pushed an updated version, see if it works for you now. – pixelou Feb 28 '16 at 18:31