Restricted Boltzmann Machine : how is it used in machine learning?

Question

Background:

Yes, Restricted Boltzmann Machine (RBM) CAN be used to initiate the weights of a neural network. Also it CAN be used in a "layer-by-layer" way to build a deep belief network (that is, to train a $n$-th layer on the top of $(n-1)$-th layer, and then to train the $n+1$-th layer on the top of the $n$-th layer, rinse and repeat ... ).

Regarding how to use RBM, details can be found from the thread of Good tutorial for Restricted Boltzmann Machines (RBM) where some papers and tutorials can be found.

My question would be :

• Is RBM really used in either industrial projects or academic projects
• If yes, how & which projects it is being used ?
• Any popular library (such as tensorflow, Caffe, Theono, etc) provides RBM module ?

Thanks for sharing. I wish to know whether RBM is really useful in practice.

Answer 1

RBM was one of the first practical ways of training/learning a deep network, having more than just one or two layers. And the deep belief network was proposed by Geoffrey Hinton, who's considered one of the 'father's of deep learning, I suppose, although Yann LeCun is the other main 'father' of deep learning, I think, or thats how I see it. Of course, everything was already invented years ago by Jurgen Schmidhuber :-)

So, RBMs are famous because 1. one of the first ways of doing deep learning 2. Geoffrey Hinton.

However, in practice, they are surely used, and useable, in academic research, since there are lots of people trying to find some unique niche, that they can be expert in, and being the worldwide expert in some niche of RBMs is a good niche as any other. However, in practice, in industry, whilst I wont claim they're never used, but they come up extremely rarely. There are simply so many very standard techniques, that train really fast and easily, like logistic regression, and feed-forward convolutional neural networks. For unsupervised, things like GANs are really popular at the moment.

Answer 2

It is possible to use RBMs to deal with typical problems that arise in data collection (that could be used for example to train a machine learning model). Such problems include imbalanced data sets (in a classification problem), or datasets with missing values (the values of some features are unknown). In the first case it is possible to train an RBM with data from the minority class and use it to generate examples for this class while in the second case it is possible to train a RBM separately for each class and uncover unknown feature values.

Another typical application of RBMs is collaborative filtering (http://dl.acm.org/citation.cfm?id=1273596).

As far as popular libraries are concerned I think deeplearning4j is a good example (http://deeplearning4j.org).