Why do we not interpret main effects if interaction terms are significant in ANOVA?

Question

I'm reading an online guide on two-way ANOVA and it says here we do not interpret the main effects if the interaction term is significant.
https://online.stat.psu.edu/stat502/lesson/4/4.1/4.1.1

Why is this the case? Would we not utilize the coefficients of the main effects in conjunction with the coefficient of the interaction term for final interpretation? For example, could we not say, given that factor 2 is _____, the deviation from the grand mean for factor 1 is ______. You get the final deviation from the grand mean by adding the two terms together.

Thanks!

Answer 1

Suppose that we have the following regression relationship:

$y=\beta_0 + \beta_1 X + \beta_2 Z + \beta_3 X \times Z + \varepsilon$.

If there is no the interaction term, i.e., $y=\beta_0 + \beta_1 X + \beta_2 Z + \varepsilon$, we can interpret the main effect as usual: "Keeping other variable, changing one unit in $X$ associates with $\beta_1$ units in $Y$".

That is not true if there is the interaction term. It is because the effect of $X$ depends on the value of $Z$ (through the interaction). Indeed, we can re-write the first formula as follows:

$y=\beta_0 + \beta_2 Z + (\beta_1 + \beta_3 Z) X + \varepsilon$.

Now we see that the coefficient of $X$ is $(\beta_1 + \beta_3 Z)$. After fixing $Z$ at a known value, we can interpret the effect of $X$ as usual. For example, with $Z=1$, the effect is represented by $\beta_1 + \beta_3$. Please note that significance of $\beta_1$ and $\beta_3$ does not guarantee a significant effect of $X$ (with $Z=1$). We need to test for the sum of those coefficients in this case.

Answer 2

This is something which is pretty well discussed in chapter 8 of John Fox's book, Applied Regression Analysis and Generalized Linear Models, or Weisberg's Applied Linear Regression. Both emphasize that your question is related to Nelder's (1977) principle of marginality.

From this last book for example:

The approach to testing we adopt in this book follows from the marginality principle suggested by Nelder (1977). A lower-order term, such as the A main effect, is never tested in models that include any of its higher-order relatives like A:B, A:C, or A:B:C. [...] An analysis of variance table derived under the marginality principle has the unfortunate name of Type II analysis of variance. [...] Type III analysis of variance violates the marginality principle. It computes the test for every regressor adjusted for every other regressor; so, for example, the test for the A main effect would include the interactions A:B, A:C, and A:B:C.

The key point is that, with "type II" ANOVA, the $F$-tests based on the sum of squares used in this decomposition are valid (i.e., do really test main effects) only when interaction is absent.

Type III ANOVA allows for testing main effects in all cases, but do ask a different research question, and should not be used carelessly.

As an intuitive answer however, the idea of non interpreting main effects when interaction terms are significant could be the following: if A:B is significant, then both A and B do play an important role in the process. Furthermore, in many instances where we can observe complex interaction patterns, asking for main effects of A and B can be simply meaningless, since the expression of A depends too much on the expression of B. (For example, let's imagine a fertilizer that would increase yields only on very wet soils, but that would drastically decrease yields on dry soils. There would be a strong interaction fertilizer:irrigation, but it would be tricky to talk about the "main effect" of this fertilizer: this simply depends too much on watering.)