Using the mean of the leaf nodes as a prediction in R

Question

I am trying to fit a random forest model to predict a binary variable. In this case, I would like to use the mean value of the response variable in each of the leaf nodes (on the train data) as the predictors for that specific tree (which will be a value between 0 and 1 depending on the observations that reached the leaf node). Generally, it takes the mode of the response variable in the leaf nodes as a prediction (so the prediction is either 0 and 1). Then, taking the average of all the tree predictions, it reaches the final prediction.

Is there an easy way to do this in R? Although this seems like a relatively straightforward question, I cannot find a solution to do this.

Answer 1

Fit a random forest for regression. I.e., make the response variable a 0-1 coded variable. Then the predicted values in the terminal nodes will be means ranging between 0 and 1 (corresponding to proportions). The final prediction will be the average of these proportions (not a majority vote).

See R example below: RF1 is a classification forest, where predictions of trees are class labels and the final prediction is a majority vote over these class labels. RF2 is a regression forest, where predictions of trees are proportions and the final prediction is the mean of these proportions. (Note that ntree = 10 is used to keep the output understandable.)

> ## Generate toy data with binary outcome:
> set.seed(42)
> x1 <- rnorm(100)
> x2 <- rnorm(100)
> y <- ifelse(x1 + x2 + rnorm(100) > 0, 1, 0)
> 
> ## Fit random forest:
> library("randomForest")
> set.seed(43)
> RF1 <- randomForest(factor(y) ~ x1 + x2, ntree = 10) # RF for classification
> RF2 <- randomForest(y ~ x1 + x2, ntree = 10) ## RF for regression
Warning message:
In randomForest.default(m, y, ...) :
  The response has five or fewer unique values.  Are you sure you want to do regression?
> testdata <- data.frame(x1 = -1, x2 = 1)
> predict(RF1, newdata = testdata, predict.all = TRUE)
$aggregate
1 
0 
Levels: 0 1

$individual
  [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
1 "0"  "0"  "0"  "1"  "1"  "0"  "0"  "1"  "0"  "1"  

> predict(RF2, newdata = testdata, predict.all = TRUE)
$aggregate
        1 
0.7666667 

$individual
  [,1]      [,2] [,3] [,4] [,5]      [,6] [,7] [,8] [,9] [,10]
1    0 0.3333333    1    1    1 0.3333333    1    1    1     1