2
$\begingroup$

My data contains 3 treatments: A (n = 10), B (n = 15) and C (n = 5). I run an ANOVA test and reject the null hypothesis, so I perform a Tukey-HSD post-hoc test and find that there is a significant difference between groups A and B.

Later, I added a new treatment group: D (n = 100). I run another ANOVA test and reject the null; then I follow up with Tukey-HSD like I did before. However, this time there is no significant difference between groups A and B. Why is this? I thought since Tukey-HSD is a pairwise comparison, then it doesn't matter if I added a new treatment group?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

P-values reported by Tukey-HSD test control for the probability of one false positive in ALL pairwise comparisons performed. Thus, the larger the number of groups, the larger the adjusted p-value.

In other words, the more treatment groups you have, the larger the probability of a single false positive since you are now performing more tests.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.