4
$\begingroup$

I'm using the following function to calculate Edwards R^2 (formula 19 in Edwards et al. 2008) of a mixed effects model (I hope the implementation is correct):

r2lmer <- function(model) {
   require(aod) # need the aod package for wald.test function
   if (class(model) != "mer") stop("mer object expected")
   n <- model@dims[['n']] # the number of observations
   p <- model@dims[['p']] # number of parameters
   df.mod <- n - p

   wald.model <- wald.test(b = fixef(model), Sigma = vcov(model), df = df.mod, Terms = 2:p)
   wald.F <- as.numeric(wald.model$result$Ftest[1])
   ( (p - 1) * 1 / df.mod * wald.F ) / (1 + (p - 1 ) * 1 / df.mod * wald.F )
}

In order to internally valdiate a mixed effects model, I would like to estimate the variance explained when the model is applied to a new data set, that is, when only the data changes, but not the parameter estimates. I'm not sure if it's even theoretically possible. Any help is highly appreciated.

Edit

If it's not a good idea to use an R^2 statistic for LMM, what other performance measures could one use to internally validate LMM? I have a LMM with just one random effects factor (varying intercept due to repeated measures) and several fixed effects factors. I'am primarily interested in the fixed effects and it is my understanding that Edward's R^2 is a good measure for the variance explained by the fixed effects. Edward's R^2 is also recommended for cross-validation in this paper:

Cheng et al. Real longitudinal data analysis for real people: Building a good enough mixed model. Statistics in Medicine. 2010, 29 504-520

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

I tend to be more and more convinced that this just generally isn't a good idea because the meaning the R^2 isn't really exactly the same as in a conventional linear regression. As such, one always runs into interpretation issues and it often distracts from the meat of the story. Time spent writing a good description of the fit of model is time better spent... but I could be wrong.

[A good answer to a question on how to write up the fit of an LMM model would be really handy (which a cursory search didn't show me).]

Improve this answer
$\endgroup$
1
1
$\begingroup$

You could always just calculate try to calculate R^2 from the squared covariance of Ypred and Ynew over the product of their variances: http://www.stator-afm.com/image-files/r-squared.gif

This works for any continuous model, regardless of how the parameters are estimated.

Improve this answer
$\endgroup$
3
  • $\begingroup$ in your posted image, which is pred and new (ie true Y on the new data set)? $\endgroup$
    – B_Miner
    Apr 18, 2011 at 18:42
  • $\begingroup$ It doesn't matter. $\endgroup$
    – Patrick McCann
    Apr 29, 2011 at 23:17
  • $\begingroup$ Link is broken, so this answer doesn't really contributes any longer $\endgroup$
    – Joris Meys
    Mar 4, 2017 at 14:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.