Can someone tell me why this is wrong:

I want to estimate the goodness-of-fit for a linear mixed effects model. I regress the fitted values against the raw values and calculate an $R^2$.

I suppose an accepted alternative would be to run the model with just the random effect and no fixed effects and compare AIC scores, but I'm specifically interested in the scenario above.


Well $R^2$ is a measure of goodness of fit for any linear model. It is the sum of squares due to the fitted model divided by the total sum of squares (denominator just adds the sum of squares due to the error term). But it is usually not used to assess the quality of the model because as in your scenario since all the dat is used to fit you could be overfitting the data. $R^2$ cannot determine overfitting. Add a variable that is just noise and the $R^2$ will still increase. Criteria like AIC and BIC are better at assessing the model because they penalize the likelihood function for having too many parameters while $R^2$ does not. Adjusted $R^2$ tries to remedy this. But I think it is not used because these other criteria tand to work better.

  • $\begingroup$ Ah, I see. I was reporting AIC scores of the reduced model anyway but just curious why this was also not appropriate. Thank you! $\endgroup$ – jslefche May 24 '12 at 19:05
  • $\begingroup$ Comparing AIC scores is an accepted way to do variable subset selection> You can compare the AIC for the full model versus the reduced. I think you should to this for the full mixed model not just the random effects. If the max AIC is not at the reduced model then the full model is considered to be the better of the two. $\endgroup$ – Michael R. Chernick May 24 '12 at 19:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.