I am looking to know what to do with bagging when n (observations) << p (features), or we have wide data. Note that each of the features are useful/significant/required. So I cannot subset my features.
Bagging, as it stands, addresses the problem of high variance in decision trees by creating multiple trees from bootstrapped observations. So when n << p, we really do not care whether each individual tree overfits, right? So bagging should work just fine with wide data. Or is there something we need to do, be careful about?
Appreciate your discussion on this idea.
This is not a textbook question