This question is more specific to machine learning. Is sampling with replacement good for random forests because it leaves some out of bag samples for testing or is it because it creates datasets/decision trees that have more variation? Or is it a mixture of both the facts.
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
We use sampling with replacement because we use bootstrap. Bootstrap imitates how we sampled the data from the population. When sampling with replacement, we end up with a sample of the same size as your original data. What bootstrap does by this, is it lets you imitate the data generating process, the underlying distribution of the data, and the variability. When sampling without replacement you would either use smaller samples or just permute the data, assuming that it is independent and identically distributed, this would lead to the same sample.
-
$\begingroup$ You use sampling with replacement when you use bootstrap. That is because you already know everything about the original sample that you are resampling from, and you want to attempt to consider what might have happened with other samples. $\endgroup$– HenrySep 19, 2021 at 15:51
-
$\begingroup$ Not sure about that because there are multiple ways of making incomplete sampling without replacement including leave-outs and cross-validation. $\endgroup$– CarlSep 25, 2021 at 4:19