R libraries for deep learning

Question

I was wondering if there's any good R libraries out there for deep learning neural networks? I know there's the nnet, neuralnet, and RSNNS, but none of these seem to implement deep learning methods.

I'm especially interested in unsupervised followed by supervised learning, and using dropout to prevent co-adaptation.

/edit: After a few years, I've found the h20 deep learning package very well-designed and easy to install. I also love the mxnet package, which is (a little) harder to install but supports things like covnets, runs on GPUs, and is really fast.

Answer 1

OpenSource h2o.deepLearning() is package for deeplearning in R from h2o.ai here's a write up http://www.r-bloggers.com/things-to-try-after-user-part-1-deep-learning-with-h2o/

And code: https://gist.github.com/woobe/3e728e02f6cc03ab86d8#file-link_data-r

######## *Convert Breast Cancer data into H2O*
dat <- BreastCancer[, -1]  # remove the ID column
dat_h2o <- as.h2o(localH2O, dat, key = 'dat')

######## *Import MNIST CSV as H2O*
dat_h2o <- h2o.importFile(localH2O, path = ".../mnist_train.csv")

######## *Using the DNN model for predictions*
h2o_yhat_test <- h2o.predict(model, test_h2o)

######## *Converting H2O format into data frame*
df_yhat_test <- as.data.frame(h2o_yhat_test)

######## Start a local cluster with 2GB RAM
library(h2o)
localH2O = h2o.init(ip = "localhost", port = 54321, startH2O = TRUE, 
                    Xmx = '2g') 
########Execute deeplearning

model <- h2o.deeplearning( x = 2:785,  # column numbers for predictors
               y = 1,   # column number for label
               data = train_h2o, # data in H2O format
               activation = "TanhWithDropout", # or 'Tanh'
               input_dropout_ratio = 0.2, # % of inputs dropout
               hidden_dropout_ratios = c(0.5,0.5,0.5), # % for nodes dropout
               balance_classes = TRUE, 
               hidden = c(50,50,50), # three layers of 50 nodes
               epochs = 100) # max. no. of epochs

Answer 2

There is a package called "darch"

http://cran.um.ac.ir/web/packages/darch/index.html

Quote from CRAN:

darch: Package for deep architectures and Restricted-Bolzmann-Machines

The darch package is build on the basis of the code from G. E. Hinton and R. R. Salakhutdinov (available under Matlab Code for deep belief nets : last visit: 01.08.2013). This package is for generating neural networks with many layers (deep architectures) and train them with the method introduced by the publications "A fast learning algorithm for deep belief nets" (G. E. Hinton, S. Osindero, Y. W. Teh) and "Reducing the dimensionality of data with neural networks" (G. E. Hinton, R. R. Salakhutdinov). This method includes a pre training with the contrastive divergence method publishing by G.E Hinton (2002) and a fine tuning with common known training algorithms like backpropagation or conjugate gradient.

Answer 3

There's another new package for deep networks in R: deepnet

I haven't tried to use it yet, but it's already been incorporated into the caret package.

Answer 4

To answer my own question, I wrote a small package in R for RBMs: https://github.com/zachmayer/rbm

This package is still under heavy development, and I know very little about RBMs, so I'd welcome any feedback (and pull requests!) you have. You can install the package using devtools:

devtools:::install_github('zachmayer/rbm')
library(rbm)
?rbm
?rbm_gpu
?stacked_rbm

The code is similar to Andrew Landgraf's implementation in R and Edwin Chen's implementation in python, but I wrote the function to be similar to the pca function in base R and include functionality for stacking. I think it's a little more user-friendly than the darch package, which I could never figure out how to use (even before it was removed from CRAN).

If you have the gputools package installed you can use your GPU for matrix operations with the rbm_gpu function. This speeds things up a lot! Furthermore, most of the work in an RBM is done with matrix operations, so just installing a good BLAS, such as openBLAS will also speed things up a lot.

Here's what happens when you run the code on Edwin's example dataset:

set.seed(10)
print('Data from: https://github.com/echen/restricted-boltzmann-machines')
Alice <- c('Harry_Potter' = 1, Avatar = 1, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #Big SF/fantasy fan.
Bob <- c('Harry_Potter' = 1, Avatar = 0, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #SF/fantasy fan, but doesn't like Avatar.
Carol <- c('Harry_Potter' = 1, Avatar = 1, 'LOTR3' = 1, Gladiator = 0, Titanic = 0, Glitter = 0) #Big SF/fantasy fan.
David <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 1, Glitter = 0) #Big Oscar winners fan.
Eric <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 0, Glitter = 0) #Oscar winners fan, except for Titanic.
Fred <- c('Harry_Potter' = 0, Avatar = 0, 'LOTR3' = 1, Gladiator = 1, Titanic = 1, Glitter = 0) #Big Oscar winners fan.
dat <- rbind(Alice, Bob, Carol, David, Eric, Fred)

#Fit a PCA model and an RBM model
PCA <- prcomp(dat, retx=TRUE)
RBM <- rbm_gpu(dat, retx=TRUE, num_hidden=2)

#Examine the 2 models
round(PCA$rotation, 2) #PCA weights
    round(RBM$rotation, 2) #RBM weights

Answer 5

You can try H2O's Deep Learning module, it is distributed and offers many advanced techniques such as dropout regularization and adaptive learning rate.

Slides: http://www.slideshare.net/0xdata/h2o-deeplearning-nextml Video: https://www.youtube.com/watch?v=gAKbAQu900w&feature=youtu.be

Tutorials: http://learn.h2o.ai Data and Scripts: http://data.h2o.ai

Documentation: http://docs.h2o.ai GitBooks: http://gitbook.io/@h2o

Answer 6

To add another answer:

mxnet is amazing, and I love it It's a little difficult to install, but it supports GPUs and multiple CPUs. If you're going to do deep learning in R (particularly on images), I highly recommend you start with mxnet.

Answer 7

While I haven't encountered a dedicated deep learning library for R, I have run into a similar discussion out on r-bloggers. The discussion centers on using RBM (Restricted Boltzman Machines). Take a look at the following link--

http://www.r-bloggers.com/restricted-boltzmann-machines-in-r/ (reposted from 'alandgraf.blogspot.com')

The author actually does a really good job of encapsulating a self-implemented algorithm in R. It must be said that I have not yet vetted the validity of the code but at least there is a glint of deep learning starting to show in R.

I hope this helps.

Answer 8

You can now also use TensorFlow from R:

https://rstudio.github.io/tensorflow/