I'm testing the randomForest package in R and observing something really strange. Let's first look at the R code below:

n <- 1000
X <- matrix(rbinom(n, 1, 0.5), n, 1)
X <- 2 * X - 1
e <- 1 / (1 + exp(X))
W <- rbinom(n, 1, e) 

dat <- cbind(W, X)
dat <- data.frame(dat)
dat$W <- factor(dat$W)

rf.classif <- randomForest(W ~ ., data = dat, ntree = 1000)
e.prob <- predict(rf.classif, newdata = dat, type = "prob")[, 2]
e.prob <- as.numeric(e.prob)

I use one single binary covariate/feature $X$ to predict $W$. As we can see, $W$ is Bernoulli distributed with probability $e$, which has value of either $0.2689414$ or $0.7310586$. I use randomForest function in R with $1000$ trees (well, since we have only one binary covariate $1$ tree should give the same result).

Anyway, the predicted probability for $W = 1$ ($e.prob$) returned by randomForest is very strange: it is either $0$ or $1$ instead of some value in between. The first $10$ values of $e.prob$ look like:

[1] 0 0 0 0 1 1 1 0 0 0

I'm trying multiple ways to resolve this problem but still getting the same result. As a side note, running a simple logistic regression would give a very good predicted probability for $W = 1$.

If Random Forest can overfit, then why cannot it give good predicted probability in this simple setting?

Any ideas/suggestions/insights would be very much appreciated.


1 Answer 1


There are two problems here. The main problem is that you are only using one predictor variable. Even though RF will use different points to construct the trees, with 1000 training points, you have a pretty dense sampling and each of the trees constructed will be almost the same. In general, RF is not for problems with a single predictor.

There is another lesser problem as well. You are testing on the training data. Since you built the trees on the data that you test, it is not surprising that they give very consistent results.

If you are just trying to test out how the probabilities work, try a problem with more predictors and generate new data from the same distribution to use for testing.

I hope that this helps.

  • $\begingroup$ Thanks for the reply! So if I have 100,000 training points, would the result be different? $\endgroup$
    – tmbtw
    Dec 13, 2016 at 18:48
  • $\begingroup$ Also, it still doesn't make sense to me why RF is not for problem with a single predictor. If it counts the values belonging to each branch in each tree, then the fraction of points with $W = 1$ should approximate the true probability. Is it correct? Last, testing on the training data is on purpose. As mentioned at the beginning, RF fails to overfit this simple problem, i.e. failing to approximate the probability in the training data well (even though it can classify well). $\endgroup$
    – tmbtw
    Dec 13, 2016 at 18:54
  • $\begingroup$ My point is that RF should work well with a single predictor theoretically, and I'm still struggling to understand why it doesn't work here. We cannot simply claim that RF doesn't work for a few predictors but it will work for many. $\endgroup$
    – tmbtw
    Dec 13, 2016 at 18:54
  • $\begingroup$ I am not saying that RF will fail. However, when it generates many trees, they will all be the same - thus, the probabilities are zero or one. With many predictors it will use different selections for different trees and so generate some variability. $\endgroup$
    – G5W
    Dec 13, 2016 at 19:58
  • $\begingroup$ When I use 1 tree, it still gives 0-1 probabilities. Why do you think that happens that way? $\endgroup$
    – tmbtw
    Dec 13, 2016 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.