Caret and randomForest number of trees [duplicate]

Question

I am puzzled as to why the caret package in R does not allow tuning on the number of trees (ntree) in a random forest (specifically in the randomForest package)? I cant imagine this is an oversight on the part of the package author - so there must be a reason for it? Can anyone shed light?

Answer 1

In theory, the performance of a RF model should be a monotonic function of ntree that plateaus beyond a certain point once you have 'enough' trees. This makes ntree more of a performance parameter than a Goldilocks parameter that you would want to tune. Caret tends to focus on tuning parameters that perform poorly for high and low values in which you want to find the happy medium.

In practice I believe there may have been studies that have found performance does reduce for very large ntree values but even if this is true the effect is subtle and requires very large forests.

There are at least 2-3 other parameters to RF that Caret doesn't tune for the same reasons as ntree.

Answer 2

Caret does let you tune the number of trees on its backend randomForest package. For instance, considering the latest version (4.6-12) as of now, you just pass the normal ntree parameter. caret will "repass" it to randomForest, e.g.:

train(formula,
      data = mydata,
      method = "rf",
      ntree = 5,
      trControl = myTrControl)

Answer 3

If you already have an idea about how many trees you want to use (Breiman recommends at least 1000) and have used randomForest::tuneRF to get an optimal mtry value (let's use 6 as an example), then:

ctrl <- trainControl(method = "none")

set.seed(2)
rforest <- train(response ~ ., data = data_set,
               method = "rf",
               ntree = 1000,
               trControl = ctrl,
               tuneGrid = data.frame(mtry = 6))

Eduardo has answered your question above but I wanted to additionally demonstrate how you can tune the value for the number of random variables used for partitioning. When tuning a random forest, this parameter has more importance than ntree as long as ntree is sufficiently large.

Answer 4

Though I agree the with the theoretical explanations posted here, in practice, having a too large number of trees is a waste of computational power and makes the model objects uncomfortably heavy for working with them (especially if you use to constantly save and load .RDS objects). Because of that, I think if we want models to be adequate we have to find somehow the minimum necessary number of trees that allow for a stable performance (and then "let the asymptotic behavior of LLN do the rest"). Perhaps if you are a very experienced statistician or if you are always working on similar problems you can use some rule of thumb (say 1000 or 10000 trees). But if your work requires you to adapt to a variety of modelling tasks, you'll end up needing some calibration method that allows you for finding an adequate and inexpensive number of trees.

For this purpose, you could just download the source code of the method from here and then rewrite it to create a custom method that adapts to your needs. Feel free to use the following example:

customRF <- list(label = "Random Forest",
 library = "randomForest",
 loop = NULL,
 type = c("Classification", "Regression"),
 parameters = data.frame(parameter = c("mtry", "ntree"), class = rep("numeric", 2), label = c("mtry", "ntree")),
 grid = function(x, y, len = NULL, search = "grid") {
   if(search == "grid") {
     out <- data.frame(mtry = caret::var_seq(p = ncol(x), 
                                             classification = is.factor(y), 
                                             len = len))
   } else {
     out <- data.frame(mtry = unique(sample(1:ncol(x), size = len, replace = TRUE)))
   }
   out
 },
 fit = function(x, y, wts, param, lev, last, classProbs, ...)
   randomForest::randomForest(x, y, mtry = param$mtry, ntree=param$ntree...),
 predict = function(modelFit, newdata, submodels = NULL)
   if(!is.null(newdata)) predict(modelFit, newdata) else predict(modelFit),
 prob = function(modelFit, newdata, submodels = NULL)
   if(!is.null(newdata)) predict(modelFit, newdata, type = "prob") else predict(modelFit, type = "prob"),
 predictors = function(x, ...) {
   ## After doing some testing, it looks like randomForest
   ## will only try to split on plain main effects (instead
   ## of interactions or terms like I(x^2).
   varIndex <- as.numeric(names(table(x$forest$bestvar)))
   varIndex <- varIndex[varIndex > 0]
   varsUsed <- names(x$forest$ncat)[varIndex]
   varsUsed
 },
 varImp = function(object, ...){
   varImp <- randomForest::importance(object, ...)
   if(object$type == "regression")
     varImp <- data.frame(Overall = varImp[,"%IncMSE"])
   else {
     retainNames <- levels(object$y)
     if(all(retainNames %in% colnames(varImp))) {
       varImp <- varImp[, retainNames]
     } else {
       varImp <- data.frame(Overall = varImp[,1])
     }
   }

   out <- as.data.frame(varImp)
   if(dim(out)[2] == 2) {
     tmp <- apply(out, 1, mean)
     out[,1] <- out[,2] <- tmp  
   }
   out
 },
 levels = function(x) x$classes,
 tags = c("Random Forest", "Ensemble Model", "Bagging", "Implicit Feature Selection"),
 sort = function(x) x[order(x[,1]),],
 oob = function(x) {
   out <- switch(x$type,
                 regression =   c(sqrt(max(x$mse[length(x$mse)], 0)), x$rsq[length(x$rsq)]),
                 classification =  c(1 - x$err.rate[x$ntree, "OOB"],
                                     e1071::classAgreement(x$confusion[,-dim(x$confusion)[2]])[["kappa"]]))
   names(out) <- if(x$type == "regression") c("RMSE", "Rsquared") else c("Accuracy", "Kappa")
   out
 })

After defining this custom method you only have to call it from train(method=customRF) and both mtry and ntree will be calibrated depending on your trainControl() specifications.