# Out-of bag error in Random Forest

I am trying to code my own, simple version of RandomForest function in R for learning purposes. However I have a hard time understanding the concept of the out-of-bag error.

Is it simply done by computing for each tree the error on sample not seen during training? So for example if my tree was build of 60% of dataset, I compute the error on remaining 40% of dataset, I repeat that logic for each tree and average the results?

Or am I misunderstanding something?

 My confusion comes from this definition:

Because this definition seems different to what my intuition says.

• Wouldn't the remaining sample be 40%? – StatsStudent Jan 19 at 17:38
• But other than that your thinking is right – peteR Jan 19 at 17:39
• BTW, this question may have already been answered on SO: stackoverflow.com/questions/18541923/… – StatsStudent Jan 19 at 17:42

You train each tree, $$T$$, with a subset of the training set (a bootstrap sample), and some of the samples are left out. So, for each tree in the forest, there are out of the bag samples. If I didn't misunderstand you, you test these tree-specific OOB samples on the tree, $$T$$, and calculate the error, do this for every tree, and average the errors, which is not the way you do normally, and in the explanation in your image. Here, you test your samples on trees, not the forest.
Just like you have a subset of samples for each tree, conversely, for each sample, $$x_i$$, you have a set of trees that your sample is out of bag, e.g. $$T_i=\{T_{i1},T_{i2},...,T_{im}\}$$ set of trees that don't contain $$x_i$$ in their sample subset. You calculate predictions of $$x_i$$ on this smaller forest, average them, which creates $$\hat{y}^{oob}$$ and then calculate your error. You do this for each sample in your dataset, and obtain the OOB error.
• But, you're using that specific sample, i.e. $x_i,y_i$; it doesn't matter if you exploit all of its features or not. Testing philosophy is based on unseen samples. – gunes Jan 21 at 19:51
• If the sample is not used in any trees, then every tree will include it in their test, (OOB) set for OOB error calculation, i.e. $T_i$ is the set of all trees. – gunes Jan 21 at 20:02