# Why is my interaction significance different depending on the model?

In this minimal reproducible example, there is an outcome variable and two predictors (age and sex).

outcome <- c(1, 2, 2, 3, 3, 4, 4, 4, 4, 5,
5, 5, 5, 5, 5, 6, 6, 7, 8, 9)

sex <- c("M","M","M","M","F","M","F","F","M","M",
"F","F","F","F","M","M","F","F","F","F")

age <- c("C","C","C","A","C","C","C","C","C","A",
"C","C","A","C","C","C","C","A","A","A")

dt <- data.frame(outcome = o, sex = as.factor(s), age = as.factor(a))


The boxplot suggests there is an interaction:

When I check the interaction as part of a model I get a different statistical significance than when I check the interaction alone.

anova(lm(outcome ~ sex + age + sex:age, dt))

anova(lm(outcome ~ sex:age, dt))


The first gives a p-value of 0.187499 for the interaction term, while the second a p-value of 0.007738.

Can someone explain the difference?

The tests are making different comparisons. In general, the tests in anova() compare the full model to the model with that term left out. (Edited to add: though things are more complicated when interactions are involved; the main effect tests being an example of that.) In the first case, the full model is

outcome ~ sex + age + sex:age


and leaving out sex:age gives

outcome ~ sex + age


the main effects model. So in that case you are really testing the interaction, and it is not significant: in the plot, it looks like C and M both give lower values than the other level (A and F respectively).

In the second case, the full model is

outcome ~ sex:age


Here sex:age is a 4 level factor containing all combinations of the factor levels. Leaving it out gives

outcome ~ 1


So in this case the test is for any kind of difference at all among the groups, and there's obviously something going on, so it comes out significant.

• Let me see if I understand: in sex + age +sex:age, the interaction term is not significant on its own because the other predictors have stronger effects, but sex:age alone is significant because it includes the sex and age factors? Dec 8 '21 at 20:59
• I'd word it differently, but you're basically right. I find it more helpful to think of what is being tested: a model with a separate effect for each of the four groups is the full model in both cases, but the comparison model is a main effects model in the first case, and a model saying all observations are the same in the second case. Dec 8 '21 at 21:02
• Thanks I've marked it as correct. Dec 8 '21 at 23:08

The reason why they are different is that in one model you included the main effects and the other model you only included the interaction term. This will yield different p-values.

These two models would be the same:

anova(lm(outcome ~ sex + age + sex:age, dt))

anova(lm(outcome ~ sex*age, dt))


Note the * notation in the model means it includes the main effects and the interaction and the : only means the interaction term.

• this is a good start, but could use more detail ... Dec 8 '21 at 20:35
• The model with formula outcome ~ sex:age includes all three terms taken as a single 3 df term representing all 4 levels of the interaction term. It's not the same as sex:age in the sex*age or sex + age + sex:age model, where it only represents the part that is independent of the main effects. Dec 8 '21 at 20:42
• Yes I am asking why including the extra terms gives a different p-value compared to the interaction alone. Thanks. Dec 8 '21 at 20:55
• I explain that in my answer. Dec 8 '21 at 20:57