Are the trees in random forest independent?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
An individual tree in a random forest is just a decision tree trained on a random subset of the full feature space. Therefore, trees are not independent of one another, as they depend on the same set of data.
Bootstrap Aggregation (i.e bagging), is a technique in which the same model is trained independently on bootstrapped samples of the full dataset. See Are observations independent in bootstrapped resamples? for a discussion of whether bootstrapped observations are independent. The TL;DR is that samples are independent with respect to the observed dataspace, but not the full dataspace.
So, single models are not independent in either technique as they depend on the full dataset.
-
$\begingroup$ reducing variance and overfit is basically the same thing. also, random forest are much more effective than simple bagging, bagging a RF wouldn't be much sensible $\endgroup$– carloCommented Jun 4, 2020 at 12:27
-
$\begingroup$ @carlo -- You are correct. Thanks for highlighting this. I think I my intention was to make a distinction between how each of the methods reduce variance (i.e you can combine the two for better effect), however, it was extremely poorly/incorrectly explained. I have removed this from my answer as it is tangential to the actual question anyways. $\endgroup$– ForrestCommented Jun 5, 2020 at 1:29