Why is the Tukey confidence interval wider than the T-test confidence interval?

Question

I have two confidence intervals for a comparison of two means. The first is a t-test CI and the other is a Tukey CI. The Tukey CI $(2.36, 26.5)$ is wider than the t-test CI $(6.46, 21.33)$.

What is the reason for this?

Answer 1

In the context of testing differences in group means on a dependent variable (DV), a confidence interval's (CI) width is proportional to the standard deviation of the DV and the critical value of the relevant test statistic. As Wikipedia explains, Tukey's HSD uses the standard deviation of the dependent variable pooled across all groups being compared in an ANOVA, whereas a $t$-test only uses the pooled $SD$ of the two groups selected for that test. Your $t$-test's pooled $SD$ may be smaller than the ANOVA's, which would produce a smaller CI for the $t$-test. When you add other groups to your ANOVA, you change the variability if the homoscedasticity assumption does not hold exactly. If the two groups you compare with a $t$-test have relatively small $SD$s, adding a third or more group(s) to your ANOVA design that have larger $SD$s (hopefully not too much larger; this can become a serious problem) will increase your mean square error ($MSE$, and thus increase the width of your CI. If you ignore those other groups with larger $SD$s in a $t$-test, you'll calculate a smaller CI, but it won't be adjusted for multiple comparisons.

If you happen to ignore other groups with smaller $SD$s rather than larger $SD$s, your confidence interval may widen, but this is less likely. As you include more groups in your ANOVA, the critical value for Tukey's HSD statistic ($q_{\rm critical}$) will increase, which also widens CIs. This is the adjustment for multiple comparisons that is the main utility of Tukey's HSD. If you leave groups out, this adjustment toward wider, more conservative CIs is reduced. Hence the effect of leaving out groups with smaller $SD$s (widening CIs) could be counteracted by the effect of reducing the number of comparisons (shrinking CIs), leaving you with CIs similar to a $t$-test (or differing in either direction, to whatever extent these effects don't balance).

In summary, there isn't enough information in the OP to guess with much accuracy, but I would guess you have at least one of these factors at play, including a third possibility:

1. The $t$-test may exclude other groups with larger $SD$s, which increase $MSE$ for Tukey's HSD.
2. The Tukey's HSD may be adjusting for several additional comparisons, increasing $q_{\rm critical}$.
3. Tukey's HSD is conservative with unbalanced-classes, which you haven't ruled out in your case.