Suppose I want to compare four non-nested models:
m0 = lm(y ~ 1) m1 = lm(y ~ x) m2 = lmer(y ~ 1 +(1|A)) m3 = lmer(y ~ x + (1|A))
Can we use AIC to compare models, even across different model types (
Yes, you can compare these models with AIC. You may have to fit m2 and m3 with maximum likelihood to do so. (The default method for lmer is REML.) Be sure to inspect graphs of model fits.
First, note that these models are, in fact, nested.
m2 is nested in
m3 (the coefficient for
x is effectively constrained to be zero in
m0 is nested in
m1 for the same reason.
m1 is nested in
m3 (with the random intercept variance for
Aset to zero in
m1. So they could be compared using a likelihood ratio test (with the mixed models being fitted with
REML = FALSE).
That said, it is also possible to compare the models using AIC (with the same proviso that the mixed models not be fitted with restricted maximum likelihood).
Note that both AIC and likelihood ratio tests have issues concerning boundary problems imposed by estimating strictly positive quantities (that is, variance components in mixed models) though this is discussed in detail in other questions and answers elsewhere on CV.