56
votes
$\begingroup$

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, neuralnet, and RSNNS, but none of these seem to implement deep learning methods.

I'm especially interested in unsupervised followed by supervised learning, and using dropout to prevent co-adaptation.

/edit: After a few years, I've found the h20 deep learning package very well-designed and easy to install. I also love the mxnet package, which is (a little) harder to install but supports things like covnets, runs on GPUs, and is really fast.

$\endgroup$

locked by whuber Feb 18 '17 at 14:31

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. See the help center for guidance on writing a good question.

Read more about locked posts here.

19
votes
$\begingroup$

OpenSource h2o.deepLearning() is package for deeplearning in R from h2o.ai here's a write up http://www.r-bloggers.com/things-to-try-after-user-part-1-deep-learning-with-h2o/

And code: https://gist.github.com/woobe/3e728e02f6cc03ab86d8#file-link_data-r

######## *Convert Breast Cancer data into H2O*
dat <- BreastCancer[, -1]  # remove the ID column
dat_h2o <- as.h2o(localH2O, dat, key = 'dat')

######## *Import MNIST CSV as H2O*
dat_h2o <- h2o.importFile(localH2O, path = ".../mnist_train.csv")

######## *Using the DNN model for predictions*
h2o_yhat_test <- h2o.predict(model, test_h2o)

######## *Converting H2O format into data frame*
df_yhat_test <- as.data.frame(h2o_yhat_test)

######## Start a local cluster with 2GB RAM
library(h2o)
localH2O = h2o.init(ip = "localhost", port = 54321, startH2O = TRUE, 
                    Xmx = '2g') 
########Execute deeplearning

model <- h2o.deeplearning( x = 2:785,  # column numbers for predictors
               y = 1,   # column number for label
               data = train_h2o, # data in H2O format
               activation = "TanhWithDropout", # or 'Tanh'
               input_dropout_ratio = 0.2, # % of inputs dropout
               hidden_dropout_ratios = c(0.5,0.5,0.5), # % for nodes dropout
               balance_classes = TRUE, 
               hidden = c(50,50,50), # three layers of 50 nodes
               epochs = 100) # max. no. of epochs
$\endgroup$
19
votes
$\begingroup$

There is a package called "darch"

http://cran.um.ac.ir/web/packages/darch/index.html

Quote from CRAN:

darch: Package for deep architectures and Restricted-Bolzmann-Machines

The darch package is build on the basis of the code from G. E. Hinton and R. R. Salakhutdinov (available under Matlab Code for deep belief nets : last visit: 01.08.2013). This package is for generating neural networks with many layers (deep architectures) and train them with the method introduced by the publications "A fast learning algorithm for deep belief nets" (G. E. Hinton, S. Osindero, Y. W. Teh) and "Reducing the dimensionality of data with neural networks" (G. E. Hinton, R. R. Salakhutdinov). This method includes a pre training with the contrastive divergence method publishing by G.E Hinton (2002) and a fine tuning with common known training algorithms like backpropagation or conjugate gradient.

$\endgroup$
  • $\begingroup$ It's just been archived! :-( $\endgroup$ – power Dec 11 '13 at 3:37
  • 3
    $\begingroup$ darch is back on CRAN! $\endgroup$ – Zach Jan 14 '14 at 19:44
  • $\begingroup$ Have you found any examples for training a deep belief network with this package and then using it to predict on new data? I find the interface it uses to be very un-intuitive. $\endgroup$ – Zach Jan 14 '14 at 22:54
  • $\begingroup$ No I haven't. Aren't there examples? If not then you could post them on this site and "answer your own question" and the score more reputation points. $\endgroup$ – power Jan 15 '14 at 6:18
  • 1
    $\begingroup$ I'll post them if I find any. So far the docs have some examples of fitting networks, but no examples of prediction. And some of the fitting examples have bugs. $\endgroup$ – Zach Jan 15 '14 at 14:03
13
votes
$\begingroup$

There's another new package for deep networks in R: deepnet

I haven't tried to use it yet, but it's already been incorporated into the caret package.

$\endgroup$
12
votes
$\begingroup$

To answer my own question, I wrote a small package in R for RBMs: https://github.com/zachmayer/rbm

This package is still under heavy development, and I know very little about RBMs, so I'd welcome any feedback (and pull requests!) you have. You can install the package using devtools:

devtools:::install_github('zachmayer/rbm')
library(rbm)
?rbm
?rbm_gpu
?stacked_rbm

The code is similar to Andrew Landgraf's implementation in R and Edwin Chen's implementation in python, but I wrote the function to be similar to the pca function in base R and include functionality for stacking. I think it's a little more user-friendly than the darch package, which I could never figure out how to use (even before it was removed from CRAN).

If you have the gputools package installed you can use your GPU for matrix operations with the rbm_gpu function. This speeds things up a lot! Furthermore, most of the work in an RBM is done with matrix operations, so just installing a good BLAS, such as openBLAS will also speed things up a lot.

Here's what happens when you run the code on Edwin's example dataset:

set.seed(10)
print('Data from: https://github.com/echen/restricted-boltzmann-machines')
Alice <- c('Harry_Potter' = 1, Avatar = 1, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #Big SF/fantasy fan.
Bob <- c('Harry_Potter' = 1, Avatar = 0, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #SF/fantasy fan, but doesn't like Avatar.
Carol <- c('Harry_Potter' = 1, Avatar = 1, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #Big SF/fantasy fan.
David <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 1, Glitter = 0) #Big Oscar winners fan.
Eric <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 0, Glitter = 0) #Oscar winners fan, except for Titanic.
Fred <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 1, Glitter = 0) #Big Oscar winners fan.
dat <- rbind(Alice, Bob, Carol, David, Eric, Fred)

#Fit a PCA model and an RBM model
PCA <- prcomp(dat, retx=TRUE)
RBM <- rbm_gpu(dat, retx=TRUE, num_hidden=2)

#Examine the 2 models
round(PCA$rotation, 2) #PCA weights
    round(RBM$rotation, 2) #RBM weights
$\endgroup$
  • 1
    $\begingroup$ If you are using PCA, try propack.svd() from the svd package. $\endgroup$ – power Feb 12 '14 at 6:58
  • $\begingroup$ @power: I'm just using PCA as a comparison, but thanks for the tip. irlba is also an excellent package for doing svd. $\endgroup$ – Zach Feb 12 '14 at 13:40
  • $\begingroup$ Does your new package provide the "dropout" training? $\endgroup$ – DavideChicco.it Jan 16 '15 at 16:36
  • $\begingroup$ @DavideChicco.it Yes, take a look at the helpfile for ?rbm. Note that rbm's are unsupervised. $\endgroup$ – Zach Jan 16 '15 at 18:06
  • $\begingroup$ Zach does this incorporate Recurrent Neural Networks? This is a big area for time series that I was looking at moving to Python for. $\endgroup$ – Hidden Markov Model Feb 26 '16 at 18:32
11
votes
$\begingroup$

You can try H2O's Deep Learning module, it is distributed and offers many advanced techniques such as dropout regularization and adaptive learning rate.

Slides: http://www.slideshare.net/0xdata/h2o-deeplearning-nextml Video: https://www.youtube.com/watch?v=gAKbAQu900w&feature=youtu.be

Tutorials: http://learn.h2o.ai Data and Scripts: http://data.h2o.ai

Documentation: http://docs.h2o.ai GitBooks: http://gitbook.io/@h2o

$\endgroup$
  • $\begingroup$ If you (the editor) are the same person as the original answerer, please merge your accounts. Then you will be able to edit your own posts. You can find out about merging your accounts in our help center. $\endgroup$ – gung - Reinstate Monica Jan 31 '15 at 20:16
6
votes
$\begingroup$

To add another answer:

mxnet is amazing, and I love it It's a little difficult to install, but it supports GPUs and multiple CPUs. If you're going to do deep learning in R (particularly on images), I highly recommend you start with mxnet.

$\endgroup$
  • $\begingroup$ +1, yes mxnet is totally implemented by C++/CUDA so it's very efficient! $\endgroup$ – Patric Apr 4 '16 at 11:46
5
votes
$\begingroup$

While I haven't encountered a dedicated deep learning library for R, I have run into a similar discussion out on r-bloggers. The discussion centers on using RBM (Restricted Boltzman Machines). Take a look at the following link--

http://www.r-bloggers.com/restricted-boltzmann-machines-in-r/ (reposted from 'alandgraf.blogspot.com')

The author actually does a really good job of encapsulating a self-implemented algorithm in R. It must be said that I have not yet vetted the validity of the code but at least there is a glint of deep learning starting to show in R.

I hope this helps.

$\endgroup$
  • $\begingroup$ I've seen this code too-- thanks for linking to it. It's good to see RBMs starting to show up in R, but I feel like R is still years behind python in terms of deep learning. I'd love to see some full featured libraries for modern neural nets! $\endgroup$ – Zach May 8 '13 at 21:56
  • $\begingroup$ I hear you there Zach. Im looking forward to getting deeper into Hinton's Neural Net course on Coursera. The allure of Theano is pushing me headlong into Python again. $\endgroup$ – Ardenne May 9 '13 at 18:16
  • $\begingroup$ Exactly. Theano is very alluring! $\endgroup$ – Zach May 9 '13 at 19:14
  • $\begingroup$ It seems that the gputools package implements some of the gpu matrix operations found in theaono: cran.r-project.org/web/packages/gputools/index.html $\endgroup$ – Zach Jan 14 '14 at 19:47
  • $\begingroup$ @Zach you almost don't need gputools and consider use preload trick on Linux to accelerate GEMM, here. $\endgroup$ – Patric Apr 4 '16 at 11:44
4
votes
$\begingroup$

You can now also use TensorFlow from R:

https://rstudio.github.io/tensorflow/

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.