1
$\begingroup$

I'm a stats consumer but by no means a statistician. I'm a little confused about how resampling an existing sample can bring useful information, but I get the general idea that you can generate confidence intervals based on the bootstrap distribution. Specifically, I'm wondering about the actual process of resampling a finite amount of times.

1) If we already know the frequency of observations in our sample, shouldn't we be able to use that probability to calculate what we expect the bootstrap distribution to look like with infinite resamples? Such that we don't actually need to run simulations of resampling? In a dumbed-down analogous way, we don't have to flip a coin 10000+ times to confirm that what the distribution of coin flips looks like.

2) Is there such a thing as too many bootstrap simulations? Generally more samples are a good thing, and I've read about different arguments for the minimum accepted number of bootstrap simulations. In regards to 1), perhaps there's a reason we wouldn't want to general to infinity simulations?

$\endgroup$
  • 2
    $\begingroup$ For nonparametric boostrapping you never need infinitely many simulations, because there exist only finitely many distinct resamples. The problem is that except in toy situations with tiny datasets (see my post at stats.stackexchange.com/a/26631/919 for a detailed example), the number of resamples is so large that it's rarely possible to compute the resampling distribution of most statistics. $\endgroup$ – whuber Apr 26 at 22:03
1
$\begingroup$

1) If we want to make an assumption about the underlying distribution, yes, we can easily calculate from a known distribution. This is where things like the normal distribution or binomial distribution come in. The problem is that if we don't know exactly what that distribution is, bootstrapping turns out to be a fair approach. Any time you decide to estimate a confidence interval without bootstrapping, though, which is very common, you've decided that the dumbed-down approach you refer to is sufficient.

2) Too many bootstrap simulations would be the number you are unwilling to wait to compute. In practice, you are minimizing a cost function that includes a) the error in your estimates, and b) the patience of your finite human life. (a) tends to get small fairly quickly, so you stop to spare (b).

$\endgroup$
  • 1
    $\begingroup$ I am curious about the senses in which you mean "most efficient" and "fairest:" is that a technical result or just a feeling? If it's the former, could you elaborate a little? $\endgroup$ – whuber Apr 26 at 22:01
  • $\begingroup$ @whuber Good point, I'll tone it down a bit. I expect a better, more technical answer will come along but felt like a simple answer would help the OP here. $\endgroup$ – Bryan Krause Apr 26 at 22:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.