2
$\begingroup$

In this minimal reproducible example, there is an outcome variable and two predictors (age and sex).

outcome <- c(1, 2, 2, 3, 3, 4, 4, 4, 4, 5,
             5, 5, 5, 5, 5, 6, 6, 7, 8, 9)

sex <- c("M","M","M","M","F","M","F","F","M","M",
         "F","F","F","F","M","M","F","F","F","F")

age <- c("C","C","C","A","C","C","C","C","C","A",
         "C","C","A","C","C","C","C","A","A","A")

dt <- data.frame(outcome = o, sex = as.factor(s), age = as.factor(a))

The boxplot suggests there is an interaction:

The boxplot suggests there is an interaction

When I check the interaction as part of a model I get a different statistical significance than when I check the interaction alone.

anova(lm(outcome ~ sex + age + sex:age, dt))

anova(lm(outcome ~ sex:age, dt))

The first gives a p-value of 0.187499 for the interaction term, while the second a p-value of 0.007738.

Can someone explain the difference?

$\endgroup$
0
2
$\begingroup$

The tests are making different comparisons. In general, the tests in anova() compare the full model to the model with that term left out. (Edited to add: though things are more complicated when interactions are involved; the main effect tests being an example of that.) In the first case, the full model is

outcome ~ sex + age + sex:age

and leaving out sex:age gives

outcome ~ sex + age

the main effects model. So in that case you are really testing the interaction, and it is not significant: in the plot, it looks like C and M both give lower values than the other level (A and F respectively).

In the second case, the full model is

outcome ~ sex:age

Here sex:age is a 4 level factor containing all combinations of the factor levels. Leaving it out gives

outcome ~ 1

So in this case the test is for any kind of difference at all among the groups, and there's obviously something going on, so it comes out significant.

$\endgroup$
3
  • $\begingroup$ Let me see if I understand: in sex + age +sex:age, the interaction term is not significant on its own because the other predictors have stronger effects, but sex:age alone is significant because it includes the sex and age factors? $\endgroup$
    – aquaporin
    Dec 8 '21 at 20:59
  • $\begingroup$ I'd word it differently, but you're basically right. I find it more helpful to think of what is being tested: a model with a separate effect for each of the four groups is the full model in both cases, but the comparison model is a main effects model in the first case, and a model saying all observations are the same in the second case. $\endgroup$ Dec 8 '21 at 21:02
  • $\begingroup$ Thanks I've marked it as correct. $\endgroup$
    – aquaporin
    Dec 8 '21 at 23:08
1
$\begingroup$

The reason why they are different is that in one model you included the main effects and the other model you only included the interaction term. This will yield different p-values.

These two models would be the same:

anova(lm(outcome ~ sex + age + sex:age, dt))

anova(lm(outcome ~ sex*age, dt))

Note the * notation in the model means it includes the main effects and the interaction and the : only means the interaction term.

$\endgroup$
4
  • 1
    $\begingroup$ this is a good start, but could use more detail ... $\endgroup$
    – Ben Bolker
    Dec 8 '21 at 20:35
  • 1
    $\begingroup$ The model with formula outcome ~ sex:age includes all three terms taken as a single 3 df term representing all 4 levels of the interaction term. It's not the same as sex:age in the sex*age or sex + age + sex:age model, where it only represents the part that is independent of the main effects. $\endgroup$ Dec 8 '21 at 20:42
  • $\begingroup$ Yes I am asking why including the extra terms gives a different p-value compared to the interaction alone. Thanks. $\endgroup$
    – aquaporin
    Dec 8 '21 at 20:55
  • $\begingroup$ I explain that in my answer. $\endgroup$ Dec 8 '21 at 20:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.