1
$\begingroup$

When using Bootstrap Aggregating (known as bagging), does all of the data get used, or is it possible for some of the data never to make it into the bagging samples and thereby getting excluded from whatever statistical procedure that is being used.

$\endgroup$
1
$\begingroup$

Say you have some data $\{(y_i, x_i)\}$ with $i=1,\ldots,n$. A bootstrap sample is when you sample from the data above with replacement, and your sample is the same size as your original data set.

On an individual bootstrap sample, let's call it $(y^*_i,x^*_i), i=1,\ldots n$, the number of times a certain row of data shows up is a binomial random variable with parameters $n$ and $1/n$. The probability it does not show up in ONE bootstrap sample is $(n-1)/n$. Let's call that number $\theta$. Then if you have $B$ bootstrap samples, and you use independence, you get the probability of a row of data never being used in ANY of the $B$ samples to be $\theta^B$.

To answer your question, there is always some probability that a row of data does not get used. But this probability gets very small when you increase $B$.

$\endgroup$
7
  • $\begingroup$ So does all the data have to get used....? $\endgroup$ Jan 27 '16 at 22:54
  • 1
    $\begingroup$ @RustyStatistician. You use sampling with replacement. There is a chance that some observations never fall into boosted sample. $\endgroup$ Jan 27 '16 at 22:55
  • $\begingroup$ @StatsStudent thanks! That's all I wanted to know. $\endgroup$ Jan 27 '16 at 22:55
  • $\begingroup$ Note that limit (1-1/n)^n as n->infinity = 1/e . Hence, for a large dataset of n items, if you sample as it size you will get ~ 36% $\endgroup$
    – DaL
    Jan 28 '16 at 10:08
  • 1
    $\begingroup$ Yeah, $B$ shouldn't depend on this probability. You're probably after better predictions. That's a different question @EngrStudent $\endgroup$
    – Taylor
    Jan 28 '16 at 14:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.