Skip to main content
3 of 3
added 12 characters in body
User1865345
  • 10.3k
  • 12
  • 23
  • 40

What's wrong with this interpretation of a 95% confidence interval?

Note: I asked a version of this as part of another question, but I'm re-asking it as a stand-alone question with more detail.

I've been trying to come up with more intuitive/less confusing ways to accurately explain what a given (say) 95% confidence interval (e.g. "the 95% CI runs between 2.5 and 3.7") actually means. I know that it does not mean "there is a 95% chance that the true value is within this interval" (because the true value is not a random variable) but that it is correct to say that, "out of all possible 95% CIs that could have been calculated from these data 95% of them will include the true value." However, based on my interpretation of the Central Limit Theorem and frequentism, I believe the following statement should also be valid interpretation of a 95% confidence interval that ranges between A and B:

"if it were true that the true mean was the the value we actually estimated, and we replicated our study a 100 times, estimating the mean each time, then 95% of those estimates would fall between A and B."

I've been told this is wrong, but I just want to understand why it's wrong.

Here's my logic. Let me know if and where there is a problem.

(First off, to avoid having to talk about t distributions let's just say that all sample sizes in this discussion are large enough that the distinction between normal and t distributions is irrelevant).

We want to estimate some parameter $\mu$ in a given population. The CTL says that if we draw a large number of random samples from this population and generate an estimate $\hat \mu$ in each of them, these estimates will form a normal curve, with a particular standard deviation, centered on the true value $\mu$. We therefore know, for example, that 95 percent of these estimates will be within 1.96 standard deviations of the true value. Of course, we only have one of these estimates, and we don't know how big the standard deviation of this sampling distribution actually is. However, we use the standard deviation $\hat \sigma$ of $\hat \mu$ in the data we actually collected as an unbiased estimator and thus estimate the standard error as $\hat \sigma / \sqrt n$.

Now, let's assume for the sake of argument that $ \mu=\hat \mu$. (obviously this is unlikely to be the case, but it is analogous to the hypothetical we use when interpreting p values: "if the null hypothesis were true..."). Under this assumption the CLT tells us that if we again took repeated random samples, other estimates of $ \mu$ from those samples will be distributed normally around $\hat \mu$ (because $ \mu=\hat \mu$), meaning that 95% of those estimates will be within 1.96 standard deviations of $\hat \mu$. As noted above, we estimate the size of "one standard deviation" in the sampling distribution by the standard error. This allows us to can calculate the following

$$A=\hat \mu+1.96\times SE$$ $$B=\hat \mu-1.96\times SE$$

We can therefore say "if $\mu= \hat \mu$, and we replicated our study a large number of times times, estimating the $\hat \mu$ each time, then 95% of those estimates of would fall between A and B."

Of course, the formula I used was just the standard formula to calculate a 95% confidence interval around $\hat \mu$. So it seems like the sentence above should be a valid interpretation of a 95% confidence interval. If it's not, why not?