I have an experiment with a 2x2 between subject design. Let's say the two factors are A (with levels a1 and a2) and B (b1 and b2). I ran an ANOVA with the formula outcome ~ A * B, but my colleague wanted to do a Welch's test as well. The formula becomes outcome ~ condition, where condition has levels a1_b1, a2_b1, a1_b2,and a2_b2.

Is it valid or necessary to run both tests? Our research question is quite vague---"what's the effect of A and B on the outcome?"

  • 1
    $\begingroup$ Your research question is exploratory rather than confirmatory, so testing is not indicated. $\endgroup$
    – whuber
    Commented Oct 23, 2022 at 14:41
  • $\begingroup$ Just run an ANOVA, and if there are significant interaction or main effects, run further post-hoc tests. I see no reason to use the Welch test approach suggested by your colleague.... $\endgroup$
    – バシル
    Commented Oct 23, 2022 at 15:13

1 Answer 1


Your question, "what's the effect of A and B on the outcome?", may be imprecise (vague) but it is not empty.

If there question were "is there an affect of A and B on the outcome?" a simple ANOVA would be perfect. It would tell you that indeed there was a difference in one of the four groups, or there was no difference in any of the four groups. I think this is a great place to start.

This is actually a good place to start, is there an effect? If not, you need not go any further. The value is it tests them all at once.

If the outcome is that there IS a difference. The WHAT the difference is must be assessed some other way. Welch's test can help you figure out HOW which set is different and if there is a directionality. But this is how you would assess what the effective difference is.

So, you are both right!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.