I have an experimental design with 4 groups: (1) Controls, (2) Treatment A, (3) Treatment B, and (4) Both treatments A and B

The data looks like this:

Example of data

In this experiment, I want to know what Treatment A does on its own, what Treatment B does on its own, and what they do together. It seems like a classic 2-way ANOVA situation.

When I just glance at the data, it obviously looks like A has some effect, B has some effect, and A+B does something opposite. When I run the actual 2-way ANOVA, however, it only comes up with an interaction effect. The main effects of A and B are both non-significant.

But the t-tests for all of these are extremely significant (p < 0.001). Is a 2-way ANOVA not the right way to go? Is it not right to interpret the main effects here? What's going on?


1 Answer 1


Think of the average score for all those who received A (including A + B). That average is around 5. Now think of the average score for all those who didn't receive A (i.e., control and B). That average is also around 5. So you can see that there is no main effect of A.

If you do the same for B, you will find the same results. Marginally, that is, ignoring whether you receive the other treatment, each factor has no effect on the outcome.

Clearly, there is something else going on, which can be found in the interaction. What is the effect of A vs. no A for those with no B? It is to increase the score by 6. What is the effect of A vs. no A for those with B? It is to decrease the score by 6. That is the effect you are observing. Another way to describe it is that receiving exactly one treatment yields a score of 8, but receiving either no treatment or both treatments yields a score of 2.

  • $\begingroup$ So compared to Controls each individual treatment has an obvious effect. If I were to go write this up in a report, would I just report those differences with t-tests then? And not refer to the 2-way ANOVA? And then when comparing group A+B to all the other groups - then report the interaction effect of the ANOVA? $\endgroup$
    – David
    Sep 29, 2017 at 11:44
  • $\begingroup$ Something like this: "We found that treatment A resulted in a significant increase of the score (unpaired t-test, p < 0.001). We also found that treatment B resulted in a significant increase of the score (unpaired t-test, p < 0.001). When both treatments were taken together, there was a significant interaction between the treatments such that the mean was not significantly different from Controls (2-way ANOVA, F = whatever, p < 0.001)." $\endgroup$
    – David
    Sep 29, 2017 at 11:47
  • $\begingroup$ I know that "post-hoc" tests are a thing, but the term "post-hoc" makes it seem as if its secondary and after-the-fact. Here I want to clearly demonstrate that A by-itself does something, B by-itself does something, and then A+B together does something different. $\endgroup$
    – David
    Sep 29, 2017 at 12:44
  • 1
    $\begingroup$ I mean, in this case, the tests are secondary and after the fact since the first question to be answered was whether there is an interaction, which there was. A series of t-tests is essentially a post-hoc test but without controlling for multiple comparisons. I honestly think displaying an interaction plot with the results of 2x2 ANOVA alone would describe the results well enough. It's not worth interpreting the main effects of the ANOVA. You can do post-hoc tests after to see the effect of each treatment when the other is in control. $\endgroup$
    – Noah
    Sep 29, 2017 at 16:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.