Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I've been wanting to experiment with a neural network for a classification problem that I'm facing. I ran into papers that talk of RBMs. But from what I can understand, they are no different from having a multilayer neural network. Is this accurate?

Moreover I work with R and am not seeing any canned packages for RBMs. I did run into literature that talks about deep learning networks which are basically stacked RBMs but not sure if it is worth the effort to implement them in R. Would anybody have any pointers? Thanks

share|improve this question
3  
They are different than multilayer networks trained with backprop. Here are some resources for all kinds of neural nets: stats.stackexchange.com/a/36257/13669 –  Stumpy Joe Pete Oct 17 '12 at 17:23
add comment

2 Answers

up vote 13 down vote accepted

First of all RBM's are certainly different from normal Neural Nets, and when used properly they achieve much better performance. Also, training a few layers of a RBM, and then using the found weights as a starting point for a Mulitlayer NN often yields better results than simply using a Multilayer NN.

The best pointer I can think of is this course on Coursera, taught by Geoofrey Hinton, who is one of the people responsible for RBMs :

https://class.coursera.org/neuralnets-2012-001/class/index

the videos on RBMs and Denoising Autoencoders(another deep architecture you might have heard of) aren't available yet (they will be in a few weeks), but given it's Coursera and Geoff Hinton, I can bet they will be a valuable learning resource for anyone interested in the topic.

As to implementation in R, I don't know any either, but if you want to implement it, better not use pure R (unless your data is not to big). The training of an RBM takes quite a long time, and if you use pure R instead of R with C it can grow significantly.

share|improve this answer
1  
I can speak to R's performance issues, as I've written some RBMs in it. The vast majority of the computation time is spent on matrix multiplies, which do tend to be slower in R than in other languages (perhaps by a factor of 3 or 5). Re-compiling R for your own system with a customized BLAS (linear algebra library) can help a lot, as can translating the slow parts to C++ with Rcpp. Writing a one-hidden-layer RBM is actually quick enough that it probably makes sense to code it in whatever language you're most comfortable in before trying to optimize for speed. –  David J. Harris Jan 15 '13 at 4:25
    
@David J. Harris Have you shared any of your implementations in R? I'd love to take a look at them. –  Zach Jan 20 at 18:48
1  
@Zach I'm pretty busy at the moment, but I'll see what I can do about open-sourcing it in the future. You can also email me at the address in my profile for a copy of what I have if you don't mind sorting through an undocumented/half-finished project. –  David J. Harris Jan 20 at 21:50
add comment

In R you can use neuralnet and RSNNS (which provides an interface to the Stuttgart Neural Network Simulator) to fit standard multilayer neural networks, but there are differences to RBM.

Regarding implementing deep neural nets in R, I think the only worthwhile strategy would be to interface existing FOSS implementations, which is usually a much better solution than just reimplementing things on your own (I never quite understood why everyone needs to reinvent the wheel). R offers a lot of functionality to do this and you can leverage the data handling of R with the speed and ready-to-use aspects of existing solutions. For example, one might interface MDP with the Python/R interfacing capabilities, see e.g., this paper.

Edit: Andrew Landgraf from Statistically Significant provides some R Code for RBM.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.