In ensemble learning with bagging, why is it important to do bootstrap resampling (sampling with replacement) instead of just sub-sampling (sampling without replacement)?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
It's not. Sampling with replacement is not required. Two issues come up when you use subsampling without replacement instead of the usual bootstrap samples: 1. You must determine what sub-sample size to use, and 2. Out of bag observations are no longer created in the same way.
See Buja and Stuetzel, 2006, section 7 (entitled "Equivalence of Resampling With and Without Replacement in Bagging") here http://www.stat.washington.edu/wxs/Learning-papers/paper-bag.pdf for a reference.
