How to interpret ANOVA output when comparing two nested mixed-effect models?

Question

I've got two models (all variable are count variables):

frm.ct <- glmer(frm ~ age + education + socialrole +
              offset(log(words)) + (1|subkorpus), family=negative.binomial(1), 
data=daten.alle.kom)

frm.oage <- glmer(frm ~ education + socialrole +
              offset(log(words)) + (1|subkorpus), family=negative.binomial(1), 
data=daten.alle.kom)

I used this to compare them:

anova(frm.ct, frm.oage)

AIC values tell me that frm.oage is the better model, right? but what do 0.0452 and 0.8315 mean?

Answer 1

The Chisq value is the test statistic of the likelihood ratio test (LRT) being applied to the two models. This value is computed as twice the difference in the log-likelihoods of the two models (the log likelihood is in column logLik). Asymptotically, the log-likelihood ratio follows a Chi-square distribution with degrees of freedom equal to the difference in degrees of freedom of the two models; here this is $|6 - 7| = 1$ and is shown in the Chi Df column. As such, the probability of observing a test statistic as extreme as the observed (0.0452) value if the two models were equivalent can be computed from that Chi-square distribution. This probability is the value 0.8315.

You can compute this yourself as

> pchisq(0.0452, df = 1, lower.tail=FALSE)
[1] 0.8316367

(This doesn't quite match as I used the rounded values printed to the console whereas the software will have used higher precision.)

In more practical terms, this just reaffirms you interpretation of the relative merits of the two models you made via AIC. The log-likelihoods of the two models are almost exactly equal indicating the two models do a similar job of fitting the data. The LRT is telling you that you'd be very likely to observe a test statistic (Chisq) as large as the one reported if the two models provided the same fit. Hence you fail to reject the null hypothesis that the likelihoods of the two models are equivalent.

frm.oage is "better" in the sense that it does as well as the more complex model with 1 fewer parameters. The AICs of the two models differ by almost 2 AIC units; which is, from the definition of AIC, what you'd expect if you added a redundant parameter with no additional explanatory power to the model

This is not unexpected; AIC is computed from the log-likelihood of a model. All indicators of the comparison of the "fits" of the two models is that they are essentially the same.