2
$\begingroup$

As I understand it, if you fit a random forest with 100 trees, we can calculate a cumulative out of bag error by

  1. making 100 sub forests, each contained within another, such that the first forest contains the first tree, the second contains the first tree + second tree, and so on...

  2. Calculate the out of bag error for each of these forests.

I don't understand why we would do this. Is it to determine the optimal number of trees by finding the number of trees such that the out of bag error is no longer decreasing?

EDIT: Here are two implementations within Matlab and R:

http://au.mathworks.com/help/stats/treebagger.ooberror.html

http://www.inside-r.org/packages/cran/randomForestSRC/docs/rfsrc

$\endgroup$
  • $\begingroup$ I don't understand why we would do this, either, and I haven't seen this approach before. Could you give a reference? $\endgroup$ – Stephan Kolassa Oct 18 '15 at 22:48
  • $\begingroup$ Hi @StephanKolassa, please see the two links I added. $\endgroup$ – Alex Oct 18 '15 at 23:13
1
$\begingroup$

You are right that most RF implementations generate a vector of OOB errors of the whole ensemble after each single tree is added; this is in fact useful to judge how many trees is enough to stabilise the prediction. But the true reason it is gathered is more because we can, as obtaining it is not as wasteful as you think.

Precisely, OOB error is usually calculated by aggregating the number of votes per each class for each object that is OOB for a current tree. This way one doesn't have to build any sub-forest, just dump the current state of OOB which is practically free.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.