4
$\begingroup$

One day I gave a $95\%$-confidence interval to a requester who know nothing about statistics. He asked me what does it mean. Roughly, I answered "The population parameter is inside the interval $95\%$ of times". Then he asked "And when it is outside, can it be far from the interval ?".

I said no but I was a little confused. What do we know about this ? It would be interesting for instance to know the distribution of the distance from $\theta_0$ to the confidence interval conditionally to the event on which $\theta_0$ is not inside the interval. Moreover could we identify some characteristics of the data sample under which the true parameter is more likely to be to the left or to the right of the confidence interval when it is outside ? (for instance the skewness of the data sample in the case of the confidence interval about a Gaussian mean).

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ +1, nice question, Stephane. I don't know if you had seen this related question: are-all-values-within-a-95-confidence-interval-equally-likely, but if not you might want to take a look at it. $\endgroup$
    – gung - Reinstate Monica
    Oct 23, 2012 at 19:11
  • $\begingroup$ @gung Indeed, good link ! $\endgroup$
    – Stéphane Laurent
    Oct 23, 2012 at 19:23
  • 1
    $\begingroup$ This is a nice example of the fact that conventional confidence intervals don't actually tell the story that most people and most circumstances need. The relevant likelihood function will give the requester exactly what they want in an easy to understand form. $\endgroup$
    – Michael Lew
    Oct 24, 2012 at 1:54
  • $\begingroup$ @MichaelLew When there are several parameters the likelihood function is not so convenient. $\endgroup$
    – Stéphane Laurent
    Oct 25, 2012 at 12:49

2 Answers 2

Reset to default
4
$\begingroup$

Just think about the differences between the different confidence intervals you could compute. We are 1% confident that the true mean is in the area inside of a 96% confidence interval but outside of the 95% confidence interval (provided we used similar methods and the 95% interval is fully contained in the 96% interval), and so forth. This of course assumes that all the conditions/assumptions for the interval hold (no black swans).

You can demonstrate this for yourself and the requestor by simulating a large number of samples from a known distribution, calculating the intervals, and seeing how far away they are when they do not contain the true value. There are many tools that do theses simulations for teaching purposes (one such is in the TeachingDemos package for R).

Whether the interval is more likely to lie to the left or right of the true value depends on how the interval was constructed. If it is an equal tail interval then either side is equally likely. If the tails are not equal, such as for minimum length intervals, then the probabilities were part of the calculation.

Be careful of the phrasing you use to explain intervals, your explanation above could be interpreted as the true value is constantly changing and 95% of the time it is in the interval that you calculated. Generally we consider the true parameter value to be fixed and it is the interval that would change from sample to sample.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, good idea to provide also the 96%-interval. About my question about the likeliness to be to the left or to the right, I'm afraid it was not clear enough. Consider for instance the context of the classical Gaussian mean, and let A be the event A="the median of the sample is larger than the mean of the sample" and B the event B="the confidence interval does not contain the true parameter value". Given the event $A\cap B$, can we say something about the likeliness to be to the left or to the right ? Or for another event A, related to some "skewness" of the data sample ? $\endgroup$
    – Stéphane Laurent
    Oct 23, 2012 at 19:18
  • $\begingroup$ About my phrasing, this was not the purpose of the question, just a rough idea of my answer. I actually explain more precisely (and in French ;) I agree that the subject of such a sentence should be the random element (as in French). $\endgroup$
    – Stéphane Laurent
    Oct 23, 2012 at 19:22
  • $\begingroup$ Whoops. Added a comment here but meant to add it up top. Don't know how to eliminate this. $\endgroup$
    – Michael Lew
    Oct 24, 2012 at 1:54
-1
$\begingroup$

While agreeing with the other answers, your requester also needs to consider the possibility of certain answers. For example, with systolic blood pressure in humans, a practical range is probably between 50 and 200, with perhaps a few outliers down to 40 and up to 250, but outside of those ranges the person would most probably be dead first and definitely so with a range from below 20 to above 300 mm Hg, except perhaps for cases of suspended animation. For some outliers you also need to consider possible equipment calibration errors. I am not a clinician (nor a statistician for that matter) so I am not sure of the exact cut off points, but I think this makes the point in principle. Human Weight might be another example where a weight (really mass) of zero is nonsensical as is a weight of 100 tons.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.