4
$\begingroup$

I've been thinking about what exactly a 95% confidence interval means when it is calculated via bootstrapping.

The formal definition of a 95% confidence interval is something like this: "if the population is repeatedly sampled and a confidence interval is calculated after each sample, then the population parameter being estimated will be included in the confidence interval 95% of the time".

But in bootstrapping, we don't calculate a confidence interval after each resample, we just calculate a single confidence interval at the end of the all of the resamples.

So the formal definition of a 95% confidence interval isn't transferable to a 95% confidence interval calculated via bootstrapping, because we don't calculate a confidence interval after each resample.

So, like Mike Lawrence asked, surely the 95% confidence interval calculated via bootstrapping must be interpreted as this: "there is a 95% probability that the confidence interval contains the population parameter we're trying to estimate". Yes?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

No, nothing has changed about the inferential situation, just the manner in which you generate the interval with the supposed properties.

So the interpretation of an CI generated this way doesn't change -- the big question is whether it really has the coverage properties claimed for it. That is, across many such samples, would the intervals generated in this manner include the parameter the specified fraction of the time?

That's a rather trickier question; often in small samples it seems that coverage can be rather smaller than the nominal coverage. Various things may improve the coverage in smaller samples in different circumstances, but I think that would be a discussion for another question.

Improve this answer
$\endgroup$
9
  • $\begingroup$ I'm still struggling! If we resample 10,000 times, why don't we calculate 10,000 confidence intervals? $\endgroup$
    – luciano
    Commented Jun 29, 2015 at 7:26
  • $\begingroup$ The point of bootstrapping a CI is to generate an interval without relying on a distributional assumption; (i) how are you going to get an interval from a single resample; and (ii) how will having 10000 intervals help you find out the coverage properties? $\endgroup$
    – Glen_b
    Commented Jun 29, 2015 at 9:17
  • 1
    $\begingroup$ In the below link, this is how the definition of a confidence interval begins: "It means that if the same population is sampled on numerous occasions and interval estimates are made on each occasion…". This definition suggests to me that we should use the standard errors of each resampleto calculate a confidence interval and there is a 95% probability each confidence interval contains the population parameter. itl.nist.gov/div898/handbook/prc/section1/prc14.htm $\endgroup$
    – luciano
    Commented Jun 29, 2015 at 9:40
  • 1
    $\begingroup$ With the bootstrap you do not estimate the interval in each replication but the point estimate. It is the distribution of these point-estimates that is an estimate of the sampling distribution of the point estimates. From that you derive the lower and upper bounds of your confidence interval, which has exactly the same interpretation as the confidence interval estimated by other means. $\endgroup$
    – Maarten Buis
    Commented Jun 29, 2015 at 9:56
  • 1
    $\begingroup$ Ok, so bootstrapping simply tells us what the sampling distribution of the point estimate would look like if we were to repeatedly sample. But despite trying hard, I can't see how this definition of a confidence interval is wrong; "there is a 95% probability the confidence interval contains the population parameter" $\endgroup$
    – luciano
    Commented Jun 29, 2015 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.