I want to use model comparison to estimate the significance the effects used in a linear mixed effects model (using R's lmer()). In theory, I could create multiple nested models, each model removing only one factor (either a main effect or an interaction), and comparing in to the full model using R's anova(). However, since I have too many such models to create, I decided to use Anova() (Type III), because as far as my understanding goes it should be equal to all the model comparisons I intended to run "manually". However, I get different results. For instance, the result of one such model comparison, where model_final_full includes all main effects and interaction, and model_nested_Polarity is identical but removing only one factor (main effect of Polarity):

anova(model_nested_Polarity, model_final_full)

The Chi-square and p-value is very different comapred to when running Anova():

Anova(model_final_full, type="III")

With anova() I get Chi-square=52.754 and p=3.782e-13. With Anova() I get for the Polarity factor Chi-square=116.3031 and p< 2.2e-16.

What am I missing? And what is the right way of doing it then?

Many thanks in advance.


I found the function mixed() (in the afex package) that does exactly what I was hoping Anova() would do - it does automatically the model comparisons of the full model with all the possible nested models. I still don't know what Anova() does and why it is different.

  • $\begingroup$ Both p-values are very small. Why do you consider them "very different"? You would draw the same conclusion from both. $\endgroup$ – Roland Jul 20 '18 at 6:15
  • $\begingroup$ Hi Roland, thanks. For this specific factor you are right, but I have other factors in my model. And for future usages, I would like to know what is the difference between these two methods. The p-vaues are indeed very small, but they are not close. The Chisq are also very different, one is almst two times of the other. Is it because the Chi-square in both cases actually measure different things? What is the way then do to model comparison without actually performing each and every comparison by myself? $\endgroup$ – Galit Jul 21 '18 at 7:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.