1
$\begingroup$

I have a mixed effects model with one random effect and two fixed effects, both of which are of interest. There is some reason to believe that there would be an interaction between these two fixed effects.

When I run the model with both main effects and their interaction, none of them are significant. If I take out the interaction term (which I probably shouldn't have done, but I did out of curiosity), one of the effects becomes (very) significant, in a way that would be of considerable interest. It seems to me that I should go with the model with the interaction in it, but I am concerned that I am missing the effect of the variable that becomes very significant once the interaction is removed.

Should I include the interaction in the model? If I haven't provided enough information here for someone else to give accurate advice, what should I consider in order to make this decision?

(Also this is my first question, so apologies if I have asked it poorly)

Improve this question
$\endgroup$
4
  • $\begingroup$ not a duplicate, because I want to know if I should include the interaction, not whether I should use the main effects. I definitely want both main effects. $\endgroup$
    – Alec Scott
    Mar 27, 2018 at 21:13
  • $\begingroup$ Even so, I was thinking the info there might answer your question. $\endgroup$
    – rolando2
    Mar 27, 2018 at 21:32
  • 1
    $\begingroup$ What is the purpose of your analysis? Are you building a prediction model or inference? $\endgroup$
    – Aksakal
    Mar 28, 2018 at 3:57
  • $\begingroup$ @Aksakal, I am using it for inference $\endgroup$
    – Alec Scott
    Mar 29, 2018 at 4:14

1 Answer 1

Reset to default
3
$\begingroup$

If you exclude the interaction term between the predictor variables X1 and X2 from your model on grounds of statistical insignificance, you'll ultimately make the assumption that the effect of X1 does not depend on the values of X2 and vice versa. How reasonable is it to make this assumption based on the existing literature in your field? If it is reasonable, then you should feel comfortable with this implied assumption. If it isn't reasonable, then making this assumption may be questionable. The statistics alone might not provide sufficient guidance here, because the interaction term can be statistically insignificant even when the effect of X1 does in fact depend on X2 (but the sample size is inadequate to afford detection of the interaction effect, for example). See Clyde Schechter's answers in this thread for a nice explanation of various reasons why the interaction term can come out as statistically insignificant: https://www.statalist.org/forums/forum/general-stata-discussion/general/1315734-insignificant-interaction-term-should-we-still-look-at-the-marginal-effects.

If you include the interaction term between X1 and X2 in your model, the implicit assumption you'll be making is that the effect of X1 depends on the values of X2 and vice versa. Again, how reasonable is it to make this type of assumption based on the existing literature in your field?

Also, note that when you include the interaction term in your model, you'll be testing (a) the conditional effect of X1 when X2 = 0 and (b) the conditional effect of X2 when X1 = 0 (as explained here: https://www.theanalysisfactor.com/testing-and-dropping-interaction-terms/ and here: The main effect will be non-significant if the interaction is significant?). In other words, you will not be testing the main effects of X1 and X2, but rather their conditional effect of each predictor when the other predictor is set to zero.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.