As the title indicates, I am using the R package gamm4 for generalized additive modeling with random effects. The response for individual $i$ in group $j$, $y_{ij}$, is modeled in terms of the corresponding predictor $x_{ij}$ as

$$ y_{ij} = f(x_{ij}) + \eta_j + \varepsilon_{ij} $$

$\eta_j$ is the group $j$ random effect and $\varepsilon_{ij}$ is the residual.

I understand that the random effects model and the GAM are equivalent in some sense, but the fitted values are substantially different in the two models, and the $R^2$ is different enough to make a substantive difference in the context where I'm using it. Here is a simulated example:

x = rnorm(1000)
gr = rep(1:100,10)
re = rep(rnorm(100),10); 
e = rnorm(1000); 
y = x^2 + re + e 
m = gamm4(y ~ s(x), random=~(1|gr))
cor( fitted(m$mer), y )^2 #R^2 for lmer model. 
cor( fitted(m$gam), y )^2 #R^2 for gam model.  

Which of the fitted values is the "right" one? Of course, I hope the answer is the $mer predictions, but I'm more interested in doing this right. Any reference on this subject would be greatly appreciated.

Edit: I notice a similar dynamic comparing the fitted values for a random effects model with those from the OLS model with the same predictors. I guess this means the difference in $R^2$ is from the incorporation of random effects into the predictions. I think this is the difference between "marginal" and "conditional" $R^2$, but I'm still unsure of which is more honest to report.

  • $\begingroup$ This question is unclear at least for Cross Validated since it does not even mention which language (R?) and package you are referring to. $\endgroup$
    – Xi'an
    Mar 20, 2016 at 7:53
  • $\begingroup$ Thank you @Xi'an, I have clarified that this is in R. I previously thought the code and the tags (two of which are R packages, which gamm4 is sort of a combination of) were sufficient. $\endgroup$
    – gammer
    Mar 20, 2016 at 14:31


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.