In 'big data' settings where the number of samples $n$ may be very large (for fixed number of features), is bagging less or more effective at reducing variance? I heard the claim that it is less effective but intuitively it should be more effective, since the chance of overlap between bootstrap samples must be smaller, and so correlation between bootstrap samples is lower.

  • 2
    $\begingroup$ In a large sample, there is less variance to begin with, so the reduction might be smaller in absolute terms. $\endgroup$ May 7, 2023 at 15:37
  • $\begingroup$ Why would it be? $\endgroup$
    – Tim
    May 7, 2023 at 16:54
  • $\begingroup$ @Tim, the variance of an estimator shrinks with sample size, so in that sense it is small in a large sample. Trying another estimator with smaller variance may not deliver a great reduction in variance on an absolute scale, as the variance is small to begin with. $\endgroup$ May 7, 2023 at 17:26
  • $\begingroup$ the question isn't really about reductions in an absolute sense, it's more about whether bagging is not a good thing to try when n is bag as opposed to when n is smaller. So I think an argument that shows why the relative reduction is smaller in big data settings (jf one exists) would be ideal $\endgroup$ May 7, 2023 at 18:35


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.