# Choose random structure in LME. Should I rely on AIC?

I want to choose the best random structure for my mixed-effects model. I have compared four models: without a random part, random intercept, random intercept and slope, and random effects:

gls0<-gls(time.dep~exper*group*fat*FL, method="REML",data=test)
lme1<-lme(time.dep~exper*group*fat*FL, random=~1|ring,data=test,method="REML")
lme2<-lme(time.dep~exper*group*fat*FL, random=~exper|ring,data=test,method="REML")
lme3<-lme(time.dep~1, random=~1|ring,data=test,method="REML")
AIC(gls0, lme1, lme2, lme3)

df       AIC
gls0 17  36.34948
lme1 18  38.34948
lme2 20  36.53359
lme3  3 -60.41729


I suppose it is obvious that random effects model has the smallest AIC, since it has no parameters in the fixed part. But this model has the highest log-likelihood as well. Does it mean that my response variable depends mostly on the random variable (and if yes, how should I interpret this result, and what should I do with other explanatory variables)? Or should I choose the model without random structure at all (but then how should I deal with pseudoreplication)?

• You cannot compare AICs across models with different fixed effects when fitting with REML (i.e., the AIC of model lme3 cannot be compared with that of the other models). You will have to use method="ML" if you want to compare these models with respect to their AIC. – Wolfgang Jun 17 '14 at 21:17
• Yes, I understand, @Wolfgang. But then how can I compare other models using ML? Because as far as I know when I compare models with different random structure I should use REML. However, it doesn' solve the problem, as with ML I get: gls0 -58.82797; lme1 -56.82797; lme2 -65.54590; lme3 -66.98719 – Dmitry Jun 17 '14 at 22:17
• I am not sure I understand your confusion now. Based on the AIC, model lme3 has the lowest value, so this would be the preferred model based on this criterion, but closely followed by lme2. – Wolfgang Jun 18 '14 at 11:46
• Sorry, @Wolfgang, I'll try to be more clear. 1. I thought I have to select the best random structure using REML (Zuur 2009). But I know, that REML is good only when the fixed structure is stable. That was my first confusing point. 2. Probably, it is a stupid question, but how to interpret that the best model is random effects model? Can I still work with lme2 or I should simply say, that my optimal model is lme3? 3. According to REML estimation gls model is better than lme models. And the plot of residuals also looks better. But I do know, that my data have pseudoreplications. How could it be? – Dmitry Jun 18 '14 at 14:20
• I would not even bother with all this model selection stuff. The model with random intercepts/slopes seems to be ok and, if I understand you correctly, is based on a priori considerations (i.e., due to pseudoreplications). So, I would just leave that part alone. I also wouldn't start fiddling around with the fixed effects part. Specify your model a priori and report that. Of course, you are free to do anything beyond that, but this is exploratory and should be declared as such. – Wolfgang Jun 18 '14 at 15:47