# random forest oob error increase as more tree build

I am implementing random forest in my code, I find that when I build one tree, the oob error(mean square error since I do regression) is close to zero, while more tree is build, the oob error stablized, this is counter-intuitive, since the textbook teaches me that the oob error should decline as more trees builds, I compare my implementation with R, the my oob is a slightly less than that of R, when mtree=1000, but when mtree=1, my oob is close to zero, while R is quite big.

Simply put, my oob increase as more trees build and stabilized. R and text book shows that oob decrease as more trees build and is stabilized.

So, is there anything wrong with my implementation? how should I tune my algorithm?

• oob? ${}{}{}$ – cardinal Apr 9 '12 at 12:29
• Out of bag, i.e. measured on the subset of the data not used to fit a particular tree. – Martin O'Leary Apr 9 '12 at 13:13
• For the same data, use the same mtry and ntree, if my oob is much larger than that of results from R randomForest package, is it means my implementation has problem? – user974270 Apr 9 '12 at 14:04
• the wired results I get is that my oob does not decrease as more trees build, and when mtry=1, my oob is smallest, while mtry incraese, oob increase also, not sure where get the problem – user974270 Apr 9 '12 at 16:32
• This sounds like a bug in your implementation of oob error. It often helps to construct a small "synthetic" test case with a known structure and see how your code performs on it. – Yevgeny Apr 11 '12 at 14:00

Another claim is that random forests “cannot overfit” the data. It is certainly true that increasing $\mathcal{B}$ [the number of trees in the ensemble] does not cause the random forest sequence to overfit... However, this limit can overfit the data; the average of fully grown trees can result in too rich a model, and incur unnecessary variance. Segal (2004) demonstrates small gains in performance by controlling the depths of the individual trees grown in random forests. Our experience is that using full-grown trees seldom costs much, and results in one less tuning parameter.