1
$\begingroup$

I am trying to fit a random forest model to predict a binary variable. In this case, I would like to use the mean value of the response variable in each of the leaf nodes (on the train data) as the predictors for that specific tree (which will be a value between 0 and 1 depending on the observations that reached the leaf node). Generally, it takes the mode of the response variable in the leaf nodes as a prediction (so the prediction is either 0 and 1). Then, taking the average of all the tree predictions, it reaches the final prediction.

Is there an easy way to do this in R? Although this seems like a relatively straightforward question, I cannot find a solution to do this.

$\endgroup$
0
1
$\begingroup$

Fit a random forest for regression. I.e., make the response variable a 0-1 coded variable. Then the predicted values in the terminal nodes will be means ranging between 0 and 1 (corresponding to proportions). The final prediction will be the average of these proportions (not a majority vote).

See R example below: RF1 is a classification forest, where predictions of trees are class labels and the final prediction is a majority vote over these class labels. RF2 is a regression forest, where predictions of trees are proportions and the final prediction is the mean of these proportions. (Note that ntree = 10 is used to keep the output understandable.)

> ## Generate toy data with binary outcome:
> set.seed(42)
> x1 <- rnorm(100)
> x2 <- rnorm(100)
> y <- ifelse(x1 + x2 + rnorm(100) > 0, 1, 0)
> 
> ## Fit random forest:
> library("randomForest")
> set.seed(43)
> RF1 <- randomForest(factor(y) ~ x1 + x2, ntree = 10) # RF for classification
> RF2 <- randomForest(y ~ x1 + x2, ntree = 10) ## RF for regression
Warning message:
In randomForest.default(m, y, ...) :
  The response has five or fewer unique values.  Are you sure you want to do regression?
> testdata <- data.frame(x1 = -1, x2 = 1)
> predict(RF1, newdata = testdata, predict.all = TRUE)
$aggregate
1 
0 
Levels: 0 1

$individual
  [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
1 "0"  "0"  "0"  "1"  "1"  "0"  "0"  "1"  "0"  "1"  

> predict(RF2, newdata = testdata, predict.all = TRUE)
$aggregate
        1 
0.7666667 

$individual
  [,1]      [,2] [,3] [,4] [,5]      [,6] [,7] [,8] [,9] [,10]
1    0 0.3333333    1    1    1 0.3333333    1    1    1     1
$\endgroup$
2
  • $\begingroup$ There's no reason to edit a closed question for grammatical issues and typos. You can just leave the closed questions alone; if OP cares enough to edit the question for re-opening, they can do so in the usual way. You can filter closed questions out of a search using closed:no. $\endgroup$ – Sycorax Mar 3 at 4:35
  • 1
    $\begingroup$ @Sycorax Very helpful, thanks! $\endgroup$ – Marjolein Fokkema Mar 3 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.