0
$\begingroup$

Say we run a 3 (A: A1,A2,A3) x 3 (B: B1,B2,B3) repeated measures ANOVA. A significant p value for factor A, for example, indicates that there is at least one pair (A1-A2,A1-A3,A2-A3) in which the mean difference is statistically significant (let's assume there are no interaction effects). Normally, to determine the specific pair(s) where a significant difference exists, post-hoc tests (multiple comparisons) are used, which will include some type of correction (e.g., Bonferroni, Tukey).

There is an alternative method, which is running the ANOVA first and then running follow-up pairwise t-tests comparing factor levels. I'm wondering whether these two alternatives are equally valid from a statistical point of view.

And a second related question: what would the correct interpretation be in a scenario where a main effect is found for a particular factor (or the interaction), but post-hoc comparisons are all non-significant?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Welcome to CV, Mikel. There are some subtleties here, beginning with the fact that a significant model overall is not equivalent to the existence of any significant differences among pairs. It would likely help to understand that before considering the rest of your question. $\endgroup$
    – whuber ♦
    Commented Jul 5, 2023 at 15:47
  • $\begingroup$ Hi whuber, I assume then that a significant p value for factor A does not indicate significant mean differences between at least one pair (A1-A2,A1-A3,A2-A3) of the factor levels? I'd really like to know about the subtleties. Thanks! $\endgroup$
    – Mikel Jimenez
    Commented Jul 9, 2023 at 10:44
  • $\begingroup$ Here are some threads to start with: stats.stackexchange.com/questions/24720, stats.stackexchange.com/questions/552472, stats.stackexchange.com/questions/547107, stats.stackexchange.com/questions/192148 (read the comments). Others like these can be found with this site search. $\endgroup$
    – whuber ♦
    Commented Jul 9, 2023 at 17:05

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.