A confidence interval is basically an interval generated by an algorithm that takes the observed data as input and gives an interval as output so that 95% of the time the interval contains the true underlying parameter.

So basically it's not a property of the interval itself but of the generating algorithm.

But we could easily create an algorithm that gives the interval (-infinity, +infinity) 95% of the time and [0,0] 5% of the time. Even if we have no access to external randomness, we could just inspect the input data and decide based on some carefully designed property that would be true 95% of the time (the property would depend on the assumed model for the distribution of the observed data).

This is obviously an extreme example, but doesn't it conform to the definition of an algorithm for generating confidence intervals? If we rule this out, then what is the requirement precisely?

For a practical example, in confidence-calibration games, one has to answer several questions such as "When was Einstein born?" with a 95% confidence interval, meaning that precisely 95% of your answers should contain the true solutions. I think the best method is to "cheat" and say huge, obviously correct intervals 95% of the time and say tiny and obviously wrong intervals 5% of the time.

How can we "repair" the definition of a confidence interval?

  • 1
    $\begingroup$ Specify that the confidence interval is based on a continuously differentiable probability distribution? $\endgroup$ – naught101 Feb 26 '15 at 1:10
  • 1
    $\begingroup$ As far as I can tell, this is an open question. $\endgroup$ – shadowtalker Feb 26 '15 at 1:51

While you can form many intervals to try to capture a value with 95% confidence, the confidence interval is the one of these intervals with the minimum width. It seems like you are considering a sort of non-deterministic way to generate an interval. In this case, you could take the expected value of the width and then the standard, minimum width interval would still win out.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.