2
$\begingroup$

I've been reading a bit about the confidence intervals on Wikipedia. The section on misunderstandings says:

A 95% confidence level does not mean that for a given realized interval there is a 95% probability that the population parameter lies within the interval (i.e., a 95% probability that the interval covers the population parameter). According to the strict frequentist interpretation, once an interval is calculated, this interval either covers the parameter value or it does not; it is no longer a matter of probability. The 95% probability relates to the reliability of the estimation procedure, not to a specific calculated interval.

I read through an article making the same point. It concludes by saying:

The nature of confidence intervals is that they can encompass the true value with some chance. In our example, a confident interval tends to encompass the true value in 90% of trials. But that does not mean that for some specific interval, there is a 90% chance of finding the true value within the interval.

But I really don't understand the point of this argument. What is the difference between saying "there is a 90% probability that the true value is within our interval" and "there is a 90% probability that our interval included the true value"? Is it just a philosophical issue or can it really lead to incorrect conclusions?

Improve this question
$\endgroup$
3

1 Answer 1

Reset to default
3
$\begingroup$

There is a difference, because you have already obtained your sample and estimated your statistics. At this point there is no more probability, your confidence interval either contains the true value or it does not.

Same logic as throwing a dice, before you throw it there is a 1/6 probability of getting a 6, after you have thrown the dice there is no probability anymore, either you obtained a 6 or you did not.

For this reason you cannot claim any probabilistic results on your obtained interval, but rather on an expected frequency, if you were to repeat this experiment many times, and estimate many intervals.

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ The distinction between "probabilistic results" and "expected frequency" seems specious. The answer certainly is not what you claim in the first paragraph. See the duplicate thread for discussion of these issues. $\endgroup$
    – whuber ♦
    Jan 28, 2021 at 13:27
  • $\begingroup$ I don't get your dice analogy. If you throw a die and see 6, there is no uncertainty involved anymore. This is most certainly not the case with confidence intervals. They're more like: a die was cast, you didn't see the answer, and someone who is known to lie 10% of the time told you it was 6. In that case, would it not be fair to say that there's a 90% probability that the die showed 6? $\endgroup$
    – relatively_random
    Jan 28, 2021 at 15:06
  • $\begingroup$ @relatively_random To be more precise, you throw a dice and you don't look at the result. The die has been cast, the result has happened, you just don't know what it is. But it either happened or it didn't. $\endgroup$
    – user2974951
    Jan 28, 2021 at 15:27
  • 1
    $\begingroup$ @relatively_random your calculation that there's a 90% probability that the die showed 6 assumes that whether the person lies is independent of the rolled number and that they choose which lie to tell by randomly picking one of the five wrong numbers. For example, if they lie with probability 0.6 when a 1 is rolled and always tell the truth otherwise, and if the lie is always that a 6 was rolled, then your probability that a 6 was really rolled given that they claim it was should be 4/15 by Bayes' Theorem, not 90%. The distinction you originally asked about has a similar explanation. $\endgroup$
    – fblundun
    Jan 28, 2021 at 16:51
  • 1
    $\begingroup$ @fblundun Excellent counter-example. Love it. I feel like the misconception is actually starting to become clear. So basically, the singleton the upgraded unreliable witness gives us is still a 90% confidence interval because they give us correct numbers 90% of the time. But the probability that what they told us is correct is not 90%. $\endgroup$
    – relatively_random
    Jan 28, 2021 at 17:08

Not the answer you're looking for? Browse other questions tagged or ask your own question.