If neither x1 nor x2 is significant for y (y=x1+x2 +x3+๐). Is it necessary to make the model y = x1 + x2 + x1*x2 + x3 +๐? (In fact, the interaction term X1X2 stands out in my analysis.)
2 Answers
In short: yes. When two variables A and B are in the model together with their interaction, the interpretation of the simple effects (A and B) changes. They are interpreted as conditional effects. With the interaction term added, A would be interpreted as the effect of A when B is zero, and B would be interpreted as the effect of B when A is zero. So it is perfectly possible and normal for a significant main effect to become non-significant when an interaction is added, simply because the slopes are not significantly different at value zero. If zero is outside the range of your variables, then you simply cannot meaningfully interpret the individual A and B coeffcients in the presence of an interaction. A possible solution is either to mean center A and B, so that the conditional effect is conditioned on the mean instead of zero, or (in case A and B are categorical) use main effect coding instead of dummy coding. That way, A and B still estimate main effects.
Here is a toy example of the kind of crossover-interaction I referred to in my comment below:
A | B | Y |
---|---|---|
1 | 1 | 14 |
1 | 1 | 11 |
1 | 1 | 18 |
1 | 1 | 16 |
1 | 1 | 13 |
1 | 2 | 29 |
1 | 2 | 25 |
1 | 2 | 28 |
1 | 2 | 29 |
1 | 2 | 27 |
2 | 1 | 33 |
2 | 1 | 29 |
2 | 1 | 30 |
2 | 1 | 30 |
2 | 1 | 26 |
2 | 2 | 11 |
2 | 2 | 14 |
2 | 2 | 20 |
2 | 2 | 13 |
2 | 2 | 11 |
If you run a factorial ANOVA or multiple regression, testing for the effect of A, B and A*B, you will see that neither A nor B are significant, but their interaction is very highly significant. This is also indicated by the profile plot showing there is a crossover interaction:
In this case, you can't claim that Variable A (horizontal axis) has an overall effect, because none of its levels (1 or 2) has a consistently higher mean than the other (21, 21.70). For B, the situation is the same, it does not have a significant overall effect (22, 20.70)
But the significant interaction tells you that the effect of B is opposite for the two levels of A, or the other way around. For a silly example, imagine that the two levels of A are Normal and Overweight, and the two levels of B are "Older" and "Younger". The dependent variable, Y, is chocolate consumption. At baseline (normal weight) older kids consume more chocolate, but for overweight kids, the difference switches direction. Older kids become self-conscious about their weight, and start eating less chocolate, while younger kids don't care, or are not aware of the problem, and eat even more. If you average over Younger-Older and Normal-Overweight, you will see no effect. But the interaction is significant, and it is meaningful, you can report and interpret it.
-
$\begingroup$ Thank you for your answer. But my friend told me. If neither A nor B is significant. Then they don't have to do interaction terms anymore. Because it doesn't make sense. Does that make sense? Do you have any recommended authoritative journals? If that's the case you can continue to do the interaction term. $\endgroup$– Kin LinApr 3, 2021 at 8:08
-
1$\begingroup$ Your friend is wrong. It is perfectly possible to have a situation in which neither A nor B are significant predictors by themselves, but their interaction is significant. This can easily happen if there is a crossover interaction for example. I will change my answer above to include a toy dataset to show you $\endgroup$ Apr 3, 2021 at 11:00
-
$\begingroup$ Thank you for your answer. It means a lot to me. If you have papers supporting this, please introduce them to me. I think I'll be better able to make that case to people. Thanks again! $\endgroup$– Kin LinApr 3, 2021 at 13:41
-
$\begingroup$ I do not know of any "papers" per se, but you should read Part III of Andrew Hayes' book, Introduction to Mediation, Moderation and Conditional Process Analysis. It is the clearest and easiest to digest explanation of interactions and moderation effects I have ever read, and it explicitly talks about how the interpretation of individual main effects change with the addition of an interaction term. This is not a topic to write papers about, just a very basic idea in multiple regression. I think any introductory textbook should cover this. $\endgroup$ Apr 3, 2021 at 17:09
-
Consider the simple model
$$Y = A\times B + \varepsilon$$
where $A$ and $B$ are your effects as described above and $\varepsilon \sim N(0, \sigma^2)$.
This model has no main effect and only interaction effects. Therefore if you rejected any relationship based on a non-signifiant linear relationship would would have been mislead. That being said, fitting an interaction only model is not conventional.
You could try fitting
$$ \log Y = \log A + \log B + \varepsilon$$
this is a slightly different version of the original model. This is a linear model but if both $\log A$ and $\log B$ are significant then this suggests a multiplicative (interaction) relationship.
-
$\begingroup$ Thank you for your answer. Mine is like this. Y = x1 + x2+ x1*x2+x3+x4+๐. But if both x1 and x2 are not significant for y (y=x1+๐; Y = x2 + ๐). Is it really necessary to make the model above with interaction terms? $\endgroup$– Kin LinApr 3, 2021 at 8:33
-
$\begingroup$ Significance can change if you add in additional terms. you should, if possible, use domain knowledge to inform your decisions. Without context of what the overarching problem is, its not obvious to me what the best way forward is $\endgroup$– jckenApr 3, 2021 at 8:38