25
$\begingroup$

I've been wanting to experiment with a neural network for a classification problem that I'm facing. I ran into papers that talk of RBMs. But from what I can understand, they are no different from having a multilayer neural network. Is this accurate?

Moreover I work with R and am not seeing any canned packages for RBMs. I did run into literature that talks about deep learning networks which are basically stacked RBMs but not sure if it is worth the effort to implement them in R. Would anybody have any pointers? Thanks

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
21
$\begingroup$

First of all RBM's are certainly different from normal Neural Nets, and when used properly they achieve much better performance. Also, training a few layers of a RBM, and then using the found weights as a starting point for a Mulitlayer NN often yields better results than simply using a Multilayer NN.

The best pointer I can think of is this course on Coursera, taught by Geoffrey Hinton, who is one of the people responsible for RBMs:

https://class.coursera.org/neuralnets-2012-001/class/index

the videos on RBMs and Denoising Autoencoders are a valuable learning resource for anyone interested in the topic.

As to implementation in R, I don't know any either, but if you want to implement it, better not use pure R (unless your data is not to big). The training of an RBM takes quite a long time, and if you use pure R instead of R with C it can grow significantly.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I can speak to R's performance issues, as I've written some RBMs in it. The vast majority of the computation time is spent on matrix multiplies, which do tend to be slower in R than in other languages (perhaps by a factor of 3 or 5). Re-compiling R for your own system with a customized BLAS (linear algebra library) can help a lot, as can translating the slow parts to C++ with Rcpp. Writing a one-hidden-layer RBM is actually quick enough that it probably makes sense to code it in whatever language you're most comfortable in before trying to optimize for speed. $\endgroup$
    – David J. Harris
    Commented Jan 15, 2013 at 4:25
  • $\begingroup$ @David J. Harris Have you shared any of your implementations in R? I'd love to take a look at them. $\endgroup$
    – Zach
    Commented Jan 20, 2014 at 18:48
  • 1
    $\begingroup$ @Zach I'm pretty busy at the moment, but I'll see what I can do about open-sourcing it in the future. You can also email me at the address in my profile for a copy of what I have if you don't mind sorting through an undocumented/half-finished project. $\endgroup$
    – David J. Harris
    Commented Jan 20, 2014 at 21:50
8
$\begingroup$

In R you can use neuralnet and RSNNS (which provides an interface to the Stuttgart Neural Network Simulator) to fit standard multilayer neural networks, but there are differences to RBM.

Regarding implementing deep neural nets in R, I think the only worthwhile strategy would be to interface existing FOSS implementations, which is usually a much better solution than just reimplementing things on your own (I never quite understood why everyone needs to reinvent the wheel). R offers a lot of functionality to do this and you can leverage the data handling of R with the speed and ready-to-use aspects of existing solutions. For example, one might interface MDP with the Python/R interfacing capabilities, see e.g., this paper.

Edit: Andrew Landgraf from Statistically Significant provides some R Code for RBM.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.