3
$\begingroup$

Many sources suggest that there is a duality between confidence intervals and hypothesis testing.(*) But I'm having trouble making sense of this philosophically. The frequentist interpretation of a confidence interval is something like (per wikipedia):

Were this procedure to be repeated on multiple samples, the calculated confidence interval (which would differ for each sample) would encompass the true population parameter 90% of the time.

Yet the p-value is defined in terms of values the sample mean might take on if the null hypothesis is true. (I.e. in the one-tailed case: $p = P(\bar x\ge \bar x_{observed}\mid\mu = H_0)$).

How is it possible to manipulate a statement about a procedure that is likely to correctly bound the true population mean into a statement about the probability of the observed sample mean?

If we understand the confidence interval as characterizing the distribution of the means of samples from a population (a view the bootstrap procedure invites), then there's no problem. There's an obvious symmetry between the case in which there is a < 5% chance of the sample mean being more extreme than $H_0$, given the actual population (i.e. $H_0$ outside of the 95% CI) and the case in which there is a < 5% chance of getting a sample mean as extreme as was observed, given that the population is really centered at $H_0$ (i.e. $p<0.05$).

But this interpretation of CIs seems to be disfavored! In particular, the wikipedia article admonishes: "A confidence interval is not a range of plausible values for the sample mean, though it may be understood as an estimate of plausible values for the population parameter."

Even if the CI is in fact a range of plausible values of the sample mean, a question remains. How precisely is such a definition equivalent to the frequentist procedure definition above?

(*) A good example is this Minitab blog post:

The confidence level is equivalent to 1 – the alpha level. So, if your significance level is 0.05, the corresponding confidence level is 95%.

  • If the P value is less than your significance (alpha) level, the hypothesis test is statistically significant.
  • If the confidence interval does not contain the null hypothesis value, the results are statistically significant.
  • If the P value is less than alpha, the confidence interval will not contain the null hypothesis value.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.