# Random Forest can't overfit?

I've read some literature that random forests can't overfit. While this sounds great, it seems too good to be true. Is it possible for rf's to overfit?

• If it can fit, it can overfit. In terms of RF, think about what happens if your forest doesn't contain enough trees (say your forest is a single tree to make the effect obvious). There are more issues than this one, but this is the most obvious. – Marc Claesen Aug 25 '14 at 16:54
• I've just responded to another thread on RF that it could easily overfit if the number of predictors is large. – horaceT Jun 11 '16 at 4:07

Try for example to estimate the model $y = log(x) + \epsilon$ with a random forest. You will get an almost zero training error but a bad prediction error
• This doesn't require any paper! You can try it yourself. Take a small simple bivariate dataset, such as $x_i=1,2,\ldots,10$ and any collection of corresponding $y_i$ you care to produce. Using least squares (because this aims to reduce the variance of the residuals), fit the series of models $y=\beta_0+\beta_1 x+\beta_2 x^2 + \cdots + \beta_k x^k$ for $k=0, 1, \ldots, 9$. Each step will reduce the variance until at the last step the variance is zero. At some point, almost anyone will agree, the models have begun to overfit the data. – whuber Jun 23 '17 at 13:11