UPDATE: caret now uses foreach
internally, so this question is no longer really relevant. If you can register a working parallel backend for foreach
, caret will use it.
I have the caret package for R, and I'm interesting in using the train
function to cross-validate my models. However, I want to speed things up, and it seems that caret provides support for parallel processing. What is the best way to access this feature on a Windows machine? I have the doSMP package, but I can't figure out how to translate the foreach
function into an lapply
function, so I can pass it to the train
function.
Here is an example of what I want to do, from the train
documentation: This is exactly what I want to do, but using the doSMP
package, rather than the doMPI
package.
## A function to emulate lapply in parallel
mpiCalcs <- function(X, FUN, ...)
}
theDots <- list(...)
parLapply(theDots$cl, X, FUN)
{
library(snow)
cl <- makeCluster(5, "MPI")
## 50 bootstrap models distributed across 5 workers
mpiControl <- trainControl(workers = 5,
number = 50,
computeFunction = mpiCalcs,
computeArgs = list(cl = cl))
set.seed(1)
usingMPI <- train(medv ~ .,
data = BostonHousing,
"glmboost",
trControl = mpiControl)
Here's a version of mbq's function that uses the same variable names as the lapply documentation:
felapply <- function(X, FUN, ...) {
foreach(i=X) %dopar% {
FUN(i, ...)
}
}
x <- felapply(seq(1,10), sqrt)
y <- lapply(seq(1,10), sqrt)
all.equal(x,y)