# AIC and anova p in multilevel model, how to interpret?

I have a model with a random-nested factor, I am comparing it with a model without the random factor (to test significance of random factor) as follows:

M2<-lme(score ~ disease* time* week, random=~1|treatment/Id, method = "REML",
data = Dat1)
M3<-gls(score ~ disease * time * week, method = "REML", data = Dat1)


Comparing the two I get:

df         AIC
M2  21 -2662.715
M3  19 -2612.308


This leads me to believe M3 would be a better model (AIC closer to 0), meaning random could be taken out, or at least that M2 and M3 are different.

I check with ANOVA:

  Model   df    AIC        BIC      logLik     Test    L.Ratio    p-value
M2     1  21  -2662.715  -2550.233  1352.358
M3     2  19  -2612.308  -2510.538  1325.154   1 vs 2  54.40752   <.0001


and p is significant p<.0001.

I am not sure how to interpret this, M2 and M3 are significantly different, but does this mean random effect is a significant factor, or does it mean M3 is a better model and the therefore random effect is not significant?

The model with the lower AIC is the "better" model. In your example, model2 is the better model, and the random factor is significant.

The significant p-value means that model2 and model3 are significantly different, and if model2 contains more information than model3, that means model3 is not adequate compared to model2.

Edit: However, since your models were fitted by REML, the reported likelihood ratio test p-value is off (not enough to change the test result in your case but may for other cases); the critical value that you should compare the likelihood ratio test statistic against is 5.14 (significance of 0.05) from a mixture of chi-squared distributions.

• @biobio, just try changing the method to "ML" instead of "REML" and do the anova (F-test) again. – KarthikS Oct 11 '16 at 21:01