29
$\begingroup$

I ran a Generalized Linear Mixed Model in R and included an interaction effect between two predictors. The interaction was not significant, but the main effects (the two predictors) both were. Now many textbook examples tell me that if there is a significant effect of the interaction, the main effects cannot be interpreted. But what if your interaction is not significant?

Can I conclude that the two predictors have an effect on the response? Or is it better to run a new model where I leave out the interaction? I prefer not to do so, because I would then have to control for multiple testing.

Improve this question
$\endgroup$
3
  • $\begingroup$ yes I meant not significant $\endgroup$
    – rozemarijn
    Jun 24, 2012 at 21:46
  • 4
    $\begingroup$ If one of these answers works for you perhaps you might accept it or request a clarification. $\endgroup$
    – conjugateprior
    Jul 23, 2012 at 14:31
  • 1
    $\begingroup$ If the interaction is not significant, then you should drop it and run a regression without it. $\endgroup$
    – Aksakal
    Feb 6, 2015 at 15:56

3 Answers 3

Reset to default
27
$\begingroup$

A little niggle

'Now many textbook examples tell me that if there is a significant effect of the interaction, the main effects cannot be interpreted'

I hope that's not true. They should say that if there is an interaction term, say between X and Z called XZ, then the interpretation of the individual coefficients for X and for Z cannot be interpreted in the same way as if XZ were not present. You can definitely interpret it.

Question 2

If the interaction makes theoretical sense then there is no reason not to leave it in, unless concerns for statistical efficiency for some reason override concerns about misspecification and allowing your theory and your model to diverge.

Given that you have left it in, then interpret your model using marginal effects in the same way as if the interaction were significant. For reference, I include a link to Brambor, Clark and Golder (2006) who explain how to interpret interaction models and how to avoid the common pitfalls.

Think of it this way: you often have control variables in a model that turn out not to be significant, but you don't (or shouldn't) go chopping them out at the first sign of missing stars.

Question 1

You ask whether you can 'conclude that the two predictors have an effect on the response?' Apparently you can, but you can also do better. For the model with the interaction term you can report what effect the two predictors actually have on the dependent variable (marginal effects) in a way that is indifferent to whether the interaction is significant, or even present in the model.

The Bottom Line

If you remove the interaction you are re-specifying the model. This may be a reasonable thing to do for many reasons, some theoretical and some statistical, but making it easier to interpret the coefficients is not one of them.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Sure. And if you're in R then you may find the package effects easier than working your way through the math, and also for generalisation to more complex models. $\endgroup$
    – conjugateprior
    Jun 24, 2012 at 23:32
  • 1
    $\begingroup$ In your bottom line it depends on what you mean by 'easier'. $\endgroup$
    – John
    Jun 26, 2012 at 13:15
  • $\begingroup$ Thank you so much for the Brambor, Clark and Golder (2006) reference! It's a very sane take at explaining interaction models. Very useful at understanding how to interpret (or NOT) the coefficients in such models... BTW, the paper comes with an internet appendix: Multiplicative Interaction Models, which comes in as a very handy overview of the discussion. $\endgroup$
    – landroni
    Feb 6, 2015 at 15:45
14
$\begingroup$

If you want the unconditional main effect then yes you do want to run a new model without the interaction term because that interaction term is not allowing you to see your unconditional main effects correctly. The main effects calculated with the interaction present are different from the main effects as one typically interprets them in something like ANOVA. For example, it's possible to have a trivial and non-signficant interaction the main effects won't be apparent when the interaction is in the model.

Let's say you have two predictors, A and B. When you include the interaction term then the magnitude of A is allowed to vary depending on B and vice versa. The reported beta coefficient in the regression output for A is then just one of many possible values. The default is to use the coefficient of A for the case when B is 0 and the interaction term is 0. But, when the regression is just additive A is not allowed to vary across B and you just get the main effect of A independent of B. These can be a very different values even if the interaction is trivial because they mean different things. The additive model is the only way to really assess the main effect by itself. On the other hand, when your interaction is meaningful (theoretically, not statistically) and you want to keep it in your model then the only way to assess A is looking at it across levels of B. That's actually the kind of thing you have to consider with respect to the interaction, not whether A is significant. You can only really see whether there's an unconditional effect of A in the additive model.

So, the models are looking at very different things and this is not an issue of multiple testing. You must look at it both ways. You don't decide based on significance. The best main effect to report is from the additive model. You make a decision on including or presenting the non significant interaction based on theoretical issues, or data presentation issues, etc.

(This is not to say that there are no potential multiple testing issues here. But what they mean depends a great deal on the theory driving the tests.)

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think @rozemarijn's concern is more about 'fishing trips', i.e. running lots of models that differ a function of how the last one's stars turned out, rather than multiple testing in the technical sense $\endgroup$
    – conjugateprior
    Jun 24, 2012 at 23:15
  • 1
    $\begingroup$ You can run all the models you want. Merely calculating a model isn't a test. A test is a logical procedure, not a mathematical one. The fact that much software by default returns p-values for parameter estimates as if you had done some sort of test doesn't mean one was. $\endgroup$
    – John
    Jun 26, 2012 at 13:17
  • $\begingroup$ And to add to what was said above, one may often do tests implicitly well aware that they will fail or pass. Those tests count toward data spelunking just as much as calculated ones. $\endgroup$
    – John
    Feb 5, 2015 at 17:58
7
$\begingroup$

If the main effects are significant but not the interaction you simply interpret the main effects, as you suggested.

You do not need to run another model without the interaction (it is generally not the best advice to exclude parameters based on significance, there are many answers here discussing that). Just take the results as they are.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ Would you give the same advice in the second paragraph if the OP indicated that the interaction was not expected to occur theoretically but was included in the model as a goodness of fit test? $\endgroup$
    – whuber ♦
    Jun 24, 2012 at 14:32
  • $\begingroup$ Thank you all so much for these quick reactions. There seems to be some differences in opinion though... John argues that I do have to run a new model without the interaction effect because "The main effect calculated with the interaction present are different from the true main effects." $\endgroup$
    – rozemarijn
    Jun 24, 2012 at 21:41
  • $\begingroup$ However, Henrik argues I should not run a new model. Perhaps I can make a decision if I know why the main effect calculated with the interaction term are different from the true main effects... $\endgroup$
    – rozemarijn
    Jun 24, 2012 at 21:43
  • $\begingroup$ In reaction to whuber the interaction was expected to occur theoretically and was not included as a goodness of fit test. $\endgroup$
    – rozemarijn
    Jun 24, 2012 at 21:44
  • 1
    $\begingroup$ To elaborate a little: the key distinction is between the idea of effects from parameters. Effects are features of the model as a whole, which may or may also not be identifiable as particular parameters. When the model is linear and there are no interactions they can so identified, but when there are interactions they can't. My claim is basically that if forced to choose, as you are, that you should care more about effects than about parameters. And if you do that, you no longer care exactly how many of the latter you need to generate the former. $\endgroup$
    – conjugateprior
    Jun 24, 2012 at 23:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.