Skip to main content
added bit on reporting CIs
Source Link

As an aside, Tukey's doesn't depend on the ANOVA results being significant; you can have significant pairwise differences even when the overall ANOVA is not significant.

That is to say, if you're going to be doing Tukey-corrected pairwise comparisons, don't bother checking for overall significance first. If you only run the Tukey comparisons after getting a significant overall p-value, you are over-correcting.

(I'm confident that this is true with regular ANOVA; it's possible that with repeated measures or non-orthogonality something else happens; anyone care to chime in?)

Finally, to agree with Freya but to provide a little more guidance, instead of a post-hoc power test, a more reasonable thing to report would be the confidence intervals; they show exactly how big a difference your experiment could have detected, which is usually what people are after when they want a post-hoc power test anyway.

As an aside, Tukey's doesn't depend on the ANOVA results being significant; you can have significant pairwise differences even when the overall ANOVA is not significant.

That is to say, if you're going to be doing Tukey-corrected pairwise comparisons, don't bother checking for overall significance first. If you only run the Tukey comparisons after getting a significant overall p-value, you are over-correcting.

(I'm confident that this is true with regular ANOVA; it's possible that with repeated measures or non-orthogonality something else happens; anyone care to chime in?)

As an aside, Tukey's doesn't depend on the ANOVA results being significant; you can have significant pairwise differences even when the overall ANOVA is not significant.

That is to say, if you're going to be doing Tukey-corrected pairwise comparisons, don't bother checking for overall significance first. If you only run the Tukey comparisons after getting a significant overall p-value, you are over-correcting.

(I'm confident that this is true with regular ANOVA; it's possible that with repeated measures or non-orthogonality something else happens; anyone care to chime in?)

Finally, to agree with Freya but to provide a little more guidance, instead of a post-hoc power test, a more reasonable thing to report would be the confidence intervals; they show exactly how big a difference your experiment could have detected, which is usually what people are after when they want a post-hoc power test anyway.

Source Link

As an aside, Tukey's doesn't depend on the ANOVA results being significant; you can have significant pairwise differences even when the overall ANOVA is not significant.

That is to say, if you're going to be doing Tukey-corrected pairwise comparisons, don't bother checking for overall significance first. If you only run the Tukey comparisons after getting a significant overall p-value, you are over-correcting.

(I'm confident that this is true with regular ANOVA; it's possible that with repeated measures or non-orthogonality something else happens; anyone care to chime in?)