Why do 95% confidence intervals contain the unknown population mean 95% of the time?

Question

I am learning about the Central Limit Theorem (CLT) and confidence intervals from this online course.

I have learnt that that the CLT states that the theoretical sampling distribution of a sample statistic (such as the sample mean) will be approximately normal, have an average of the true, population-level value being estimated and have variability that is a function of the variation of individual values in the population (standard deviation) and the size of the sample the statistic is based upon.

Further, since the distribution of estimates around their truth is normal, 95% of the time, you'll get an estimate that falls ± 2 standard errors from the truth.

However, there is something that I do not understand with regards to 95% confidence intervals. I have learnt from this course that if you take your estimate and add ± 2 standard errors, you will get an interval that contains the unknown truth 95% of the time (i.e. the 95% confidence interval).

I understand that in a theoretical sampling distribution, the true value being estimated is at the center so this is why 95% of the estimates you'll get fall ± 2 standard errors from the truth (since this is a property of normal distributions).

However, the estimate is not likely to be at the center of the distribution in contrast to the truth. Why is it the case that when you add ± 2 standard errors to your estimate, you get an interval that contains the truth 95% of the time?

Answer 1

The reason why the confidence interval has this coverage property is due to some probability manipulations. It might be instructive to do some math.

The Central Limit Theorem states that as $n \to \infty$,

$$ z=\dfrac{\bar{X}- \mu}{\sigma/\sqrt{n}} \sim \mathcal{N}(0, 1) $$

Let $z_\alpha$ be the $\alpha^{th}$ quantile of a standard normal distirbution. If $\alpha>0.5$, then

$$ P( \vert z \vert \leq z_{1-\alpha/2} ) = 1-\alpha $$

and

$$ P \left( -z_{1-\alpha/2}\dfrac{\sigma}{\sqrt{n}} \leq \bar{X} - \mu \leq z_{1-\alpha/2} \dfrac{\sigma}{\sqrt{n}}\right) = 1-\alpha $$

Rearranging to put $\mu$ in the middle of the inequality

$$ P\left( \bar{X} - z_{1-\alpha/2}\dfrac{\sigma}{\sqrt{n}} \leq \mu \leq \bar{X} + z_{1-\alpha/2}\dfrac{\sigma}{\sqrt{n}} \right) = 1-\alpha$$

The argument in $P$ is familliar, right? These are the end points of a confidence interval.