How does out of bag error represent the overall performance of a random forest?

Question

From my understanding, OOB error is computed as a weighted average of the errors from each individual learner. Each decision tree is trained on 2/3 of the data and the OOB error for each learner is computed as just the error on the remaining 1/3. But the overall OOB error for the random forest is the average of OOB errors from each individual learner.

So if that is how OOB error is obtained, then how does it represent the overall ensemble and not just the individual learners if the random forest then combines those individual learners?

I'm using OOBPredict and oobError in Matlab. I wish to create a random forest for classification but I need a metric to evaluate it, but I do not know how to interpret OOB error.

Would it be better to evaluate the model by doing 10-fold cross validation while only feeding 9/10 of the data into the random forest, getting a prediction accuracy on the 1/10, and repeating? Or is OOB representative?

Answer 1

OOB is good as proxy for the generalization error, but probably you will get more accurate results, if you do a 5-fold or 10-fold cross-validation and repeat this several times. But this procedure also takes more time.

You can see the OOB Error as an estimation for each observation with all trees except of the trees where the observation was used for training. So on average exp(-1) * number_of_trees are used for each prediction of an observation. If you have learned enough trees, this is not a problem, as random forest converges, see the Breiman, 2001, paper.

Furthermore I think OOB-Error is most comparable to the Leave-One-Out Crossvalidation. For a discussion between LOOCV and k-fold-CV see here: 10-fold Cross-validation vs leave-one-out cross-validation