2
$\begingroup$

Beginner question here. I've read (see this paper) that when computing confidence intervals if you get a given interval with a confidence level of 95%, then it is a fallacy to think that the true parameter is inside that confidence interval with 95% of confidence. The paper linked above makes a case and explains this, but I'm afraid I don't get the intution.

For instance, let's say that I randomly sampled from a population 1,000,000 times, and that I'm sure that 95% of the resulting confidence intervals will contain the population parameter. So I know that 950,000 confidence intervals contain the population parameter, and that 50,000 don't.

It seems fairly reasonable to me that if I pick up one of those confidence intervals, there is a 95% chance that I will have one of the intervals that do contain the true population parameter. So, if I sampled just once from the population and calculated a confidence interval, then it will count as a random sample from the set of possible intervals. I am 95% sure that this interval contains the true population parameter.

Kristoffer Magnusson has a nice web app (here) where he seems to suggest that there is indeed a fallacy in my kind of reasoning. So several people say that this is a basic thinking flaw, but I don't really get it. Any help will be most welcome.

Improve this question
$\endgroup$
2

1 Answer 1

Reset to default
3
$\begingroup$

The reason why it is won't to say there is a 95% chance the confidence interval contains the true parameter (or the probability of the interval containing the true parameter is .95) is because the parameter is either contained in the interval or not. There are two cases here:

  1. The parameter is contained in the interval. What is the probability (or chance) the confidence interval contains the parameter? 1 since it is in the interval.

  2. The parameter is not contained in the interval. What is the probability (or chance) the confidence interval contains the parameter? 0 since it is not in the interval.

When you say "It seems fairly reasonable to me that if I pick up one of those confidence intervals, there is a 95% chance that I will have one of the intervals that do contain the true population parameter." The parameter is either in the interval or not, there is not randomness here. So it does not make sense to say there is a 95% chance the confidence interval contains the true parameter. A 95% confidence interval means exactly what you described, if you were to randomly sample the same way 1000 times and create 1000 confidence intervals, approximately 95% of these intervals would contain the true parameter.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks, Lauren. Reading your answer, and some other threads here on SO, it seems to be a rather philosophical question in the freq vs. bayesian field. But the distinction between confidence intervals (based on the idea that probability and proportions are the same thing) and credible intervals (probability as degree of rational belief) sheds some light. Thanks. $\endgroup$
    – numberfive
    Sep 26, 2015 at 4:45
  • $\begingroup$ Sure - the reply I gave is based on frequentist methods. $\endgroup$
    – Lauren Goodwin
    Sep 28, 2015 at 20:06
  • $\begingroup$ Ok. I see that what Baesianists want to tell us now. They want to tell that there is no posterior probabilities in some cases, when you are aware of the true parameter value. The fact that you are looking for the interval rather than use exact parameter value for the 'interval' suggests that you are not aware of what the value is. So, the interval is probabilistic. Otherwise, baesians should forbid everybody to speak about probabilities before the Bertulli trials. The trial either success or fails. So, there is no probability. It is either certainly 1 or 0 ;) I see the same logic here. $\endgroup$
    – Little Alien
    Aug 25, 2016 at 21:36
  • $\begingroup$ Ok, I have got it. I think that your answer is not full because it creates impression that we know the true value or parameter to be found in the CI. But, we do not know it. Yet, the paper demonstrates that baesian likelihood is a kind of interval for which we sometimes can say that it contains our parameter with 100% of probability. Yet, the CI intervals say that there is only 50% probability, which is obviously wrong. $\endgroup$
    – Little Alien
    Sep 1, 2016 at 20:46

Not the answer you're looking for? Browse other questions tagged or ask your own question.