0
$\begingroup$

I have some data with categorical predictors and I'm wondering about comparing the confidence intervals of difference in means of one pairwise comparison in my TukeyHSD analysis vs that of a two-sample t-test. Which one will have a wider confidence interval and will it always be that way for every pairwise comparison?

I just don't understand what's happening, doesn't a lower p-value mean a wider CI? And if so, then why would we be able to expect TukeyHSD to have a wider/skinnier CI if the p-value for each pairwise comparison in TukeyHSD seems to be either high or low? TukeyHSD adjusts for the probability of making type 1 errors. Bonferroni requires a lower P-value for each individual comparison to lower the family-wise error rate which I assume Tukey does as well, so if the p-value goes down, shouldn't the CI go up?

My CI's for the pairwise comparison $D-C$ for example were:

TukeyHSD: $(-16.8, -4.93)$ and

Two-sample t-test: $(-14.91, -6.82)$

So the CI for the TukeyHSD was wider. Why and will it always be like this?

Improve this question
$\endgroup$
4
  • $\begingroup$ The phrase "higher CI" is not standard and can be interpreted in various ways. $\endgroup$
    – Harvey Motulsky
    Commented Mar 30, 2023 at 1:16
  • $\begingroup$ It would help to show an example with values and calculated confidence intervals so it is clear what you are asking. $\endgroup$
    – Harvey Motulsky
    Commented Mar 30, 2023 at 1:16
  • $\begingroup$ @HarveyMotulsky How about wider? $\endgroup$
    – Cotton Headed Ninnymuggins
    Commented Mar 30, 2023 at 1:17
  • $\begingroup$ But the Tukey confidence intervals apply the 95% (or whatever) confidence level to an entire family of comparisons, not to a single comparison. So the two intervals are not really comparable. $\endgroup$
    – Harvey Motulsky
    Commented Mar 30, 2023 at 17:05

1 Answer 1

Reset to default
1
$\begingroup$

Looks like in your example t-test interval is not adjusted for multiplicity. The quote in bold implies that Bonferroni-adjusted t-test interval would be wider that TukeyHSD's - it's because Bonferroni's procedure is more crude. But indeed, without any adjustment at all t-test interval would be narrower than with smart adjustment by HSD. So raw t-test interval size < TukeyHSD's < Bonferroni-adjusted t-test interval.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.