3
$\begingroup$

I am fitting a linear mixed effect model (using lmer), and want to obtain p-values via model comparison. As far as I understand it, the procedure is: (i) create a nested model identical to the big model minus one of the variables you want to test; (ii) compare the big model to the nested model, the p-value of this comparison is attributed to the excluded variable; (iii) repeat this for each of the variables (i.e. each nested model is compared to the big model).

What happens when I test for a main effect, and the model includes an interaction term of this factor with some other factor? I learned from other threads it is wrong to have higher-level terms without lower-level ones (this one, or this one, but my mathematical understanding is limited). And yet, it seems that model comparison is standardly done to obtain p-values of main effects even when interaction terms are included (e.g. by using the function mixed() from the afex package with the LRT method, or manually constructing a nested model with an itneraction but without a main effect).

My question is actually twofold:

  1. Is it wrong to rely on model comparison for p-values of main effects when I have an interaction? Or does the function mixed() (with the LRT method) do something other than the procedure described above?

  2. Assuming all contrasts are orthogonal, I don't understand why it is wrong to have an interaction without a main effect. With orthogonal contrasts, isn't the variability explained by interaction independent of the variability of the main effects?

Improve this question
$\endgroup$
8
  • $\begingroup$ When the interaction between A and B exists, it is hard to define the "main effects" of A and B. So there is nothing to estimate/test. Of course, if you can define them, you can estimate/test them. $\endgroup$
    – user158565
    Nov 25, 2018 at 16:38
  • $\begingroup$ The main effects are defined as the mean effect across all the levels of the other effect. But how does this answer my question regarding a nested model which is identical to the big model in everything but a main effect? $\endgroup$
    – Galit
    Nov 25, 2018 at 17:48
  • $\begingroup$ Suppose 2 factors X and Y, X has level a and b, Y has level 1 and 2 and interaction exist. For response variable Z, the means for 4 different combination of X and Y are $\mu_{a1}, \mu_{a2}, \mu_{b1}, \mu_{b2}$. How to define the main effect of X and Y? $\endgroup$
    – user158565
    Nov 25, 2018 at 18:01
  • $\begingroup$ It depends on the coding, but assuming sum coding (as I did), and a balanced sample, the main effect of X is the contrast between (μa1+μa2)/2, or -(μb1+μb2)/2. Likewise, for Y: (μa1+μb1)/2, or -(μa2+μb2)/2. $\endgroup$
    – Galit
    Nov 25, 2018 at 18:48
  • $\begingroup$ if $\mu_{a1} = -1, \mu_{a2} =1, \mu_{b1} = 1, \mu_{b2} = -1$, then $(\mu_{a1}+\mu_{a2})/2 = 0$ and $-(\mu_{b1}+\mu_{b2})/2 =0$. Is it you want? Mathematically, it is correct; in practice, it always misleads. $\endgroup$
    – user158565
    Nov 25, 2018 at 19:04

1 Answer 1

Reset to default
1
$\begingroup$

I don't know what the function mixed does, but if you have an interaction then the test of the main effect is the effect when other variable(s) in the interaction are 0. It is not the usual main effect.

As to why you should include the main effects when you have an interaction, you already linked to threads that answer that question. Was there something specific in those threads that you did not understand? (You note that your "mathematical understanding is limited" - that's OK, so is mine - but it's hard to know how to help if we don't know where you are confused.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Do you mean that neither of the two models that I compare when testing for the main effect should include an interaction term? As for the second question - I saw that people answered about how an interaction without a main effect can be spuriously significant if the main effect is significant, because it "swallows" the variability of the main effect. But fi their variabilities are independent - how does that happen? When contrasts are orthogonal then each accounts for a unique amount of variability. Am I wrong? $\endgroup$
    – Galit
    Nov 25, 2018 at 11:13
  • $\begingroup$ If your model needs an interaction, then the interaction should be included. But testing for main effects then means something else. But don't drop the interaction if it is needed. As to why to include main effects, it's because the interpretation of an interaction without main effects is very problematic. Maybe I will write a blog post on this - it's a little too long for here. $\endgroup$
    – Peter Flom
    Nov 25, 2018 at 11:21
  • $\begingroup$ I'd be happy to read this blog post if ever you write it! Just to make sure that I understood your intention - (i) the nested model includes an interaction, but not a main effect (and that's fine for purposes of model comparison?). (ii) the p-value obtained by comparing these two models is not of the main effect, but of something else? .... $\endgroup$
    – Galit
    Nov 25, 2018 at 11:29
  • $\begingroup$ OK, I'll think about the blog post, but 1) No. You should always include the main effects when you include the interaction. Otherwise, the model will not capture the right thins. The estimates will be screwy. 2) If you don't include a main effect there will be no estimates of it. $\endgroup$
    – Peter Flom
    Nov 25, 2018 at 14:56
  • 1
    $\begingroup$ Hi Peter, many thank on the blog post. I left you a comment there, but I'm not sure it registered. $\endgroup$
    – Galit
    Dec 6, 2018 at 7:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.