Comparing non-nested GLMMs with AIC

Question

Suppose I want to compare four non-nested models:

m0 = lm(y ~ 1)

m1 = lm(y ~ x)

m2 = lmer(y ~ 1 +(1|A))

m3 = lmer(y ~ x + (1|A))

Can we use AIC to compare models, even across different model types (lm vs. lmer)?

Answer 1

Yes, you can compare these models with AIC. You may have to fit m2 and m3 with maximum likelihood to do so. (The default method for lmer is REML.) Be sure to inspect graphs of model fits.

Answer 2

First, note that these models are, in fact, nested. m2 is nested in m3 (the coefficient for x is effectively constrained to be zero in m2. m0 is nested in m1 for the same reason. m1 is nested in m3 (with the random intercept variance for Aset to zero in m1. So they could be compared using a likelihood ratio test (with the mixed models being fitted with REML = FALSE).

That said, it is also possible to compare the models using AIC (with the same proviso that the mixed models not be fitted with restricted maximum likelihood).

Note that both AIC and likelihood ratio tests have issues concerning boundary problems imposed by estimating strictly positive quantities (that is, variance components in mixed models) though this is discussed in detail in other questions and answers elsewhere on CV.