What are easy to interpret, goodness of fit measures for linear mixed effects models?

Question

I am currently using the R package lme4.

I am using a linear mixed effects models with random effects:

library(lme4)
mod1 <- lmer(r1 ~ (1 | site), data = sample_set) #Only random effects
mod2 <- lmer(r1 ~ p1 + (1 | site), data = sample_set) #One fixed effect + 
            # random effects
mod3 <- lmer(r1 ~ p1 + p2 + (1 | site), data = sample_set) #Two fixed effects + 
            # random effects

To compare models, I am using the anova function and looking at differences in AIC relative to the lowest AIC model:

anova(mod1, mod2, mod3)

The above is fine for comparing models.

However, I also need some simple way to interpret goodness of fit measures for each model. Does anyone have experience with such measures? I have done some research, and there are journal papers on R squared for the fixed effects of mixed effects models:

• Cheng, J., Edwards, L. J., Maldonado-Molina, M. M., Komro, K. A., & Muller, K. E. (2010). Real longitudinal data analysis for real people: Building a good enough mixed model. Statistics in Medicine, 29(4), 504-520. doi: 10.1002/sim.3775
• Edwards, L. J., Muller, K. E., Wolfinger, R. D., Qaqish, B. F., & Schabenberger, O. (2008). An R2 statistic for fixed effects in the linear mixed model. Statistics in Medicine, 27(29), 6137-6157. doi: 10.1002/sim.3429

It seems however, that there is some criticism surrounding the use of measures such as those proposed in the above papers.

Could someone please suggest a few easy to interpret, goodness of fit measures that could apply to my models?

Answer 1

There is nothing such as an easy to interpret goodness of fit measure for linear mixed models :)

Random effect fit (mod1) can be measured by ICC and ICC2 (the ratio between variance accounted by random effects and the residual variance). psychometric R package includes a function to extract them form a lme object.

It is possible to use R2 to assess fixed effect (mod2, mod3), but this can be tricky: When two models show a similar R2 it can be the case that one is more "accurate", but that is masked by its fixed factor "subtracting" a greater variance component to the random effect. On the other hand it is easy to interpret a greater R2 of the highest order model (eg mod3). In Baayen's chapter on mixed models there is a nice discussion about this. Also, it's tutorial is very clear.

A possible solution is to consider each variance component independently, and then use them to compare the models.