# Using gamm4 for generalized additive models with random effects. Use the fitted values from the mer part or the gam part?

As the title indicates, I am using the R package gamm4 for generalized additive modeling with random effects. The response for individual $i$ in group $j$, $y_{ij}$, is modeled in terms of the corresponding predictor $x_{ij}$ as

$$y_{ij} = f(x_{ij}) + \eta_j + \varepsilon_{ij}$$

$\eta_j$ is the group $j$ random effect and $\varepsilon_{ij}$ is the residual.

I understand that the random effects model and the GAM are equivalent in some sense, but the fitted values are substantially different in the two models, and the $R^2$ is different enough to make a substantive difference in the context where I'm using it. Here is a simulated example:

library(gamm4)
library(mgcv)
x = rnorm(1000)
gr = rep(1:100,10)
re = rep(rnorm(100),10);
e = rnorm(1000);
y = x^2 + re + e
m = gamm4(y ~ s(x), random=~(1|gr))
cor( fitted(m$mer), y )^2 #R^2 for lmer model. cor( fitted(m$gam), y )^2 #R^2 for gam model.


Which of the fitted values is the "right" one? Of course, I hope the answer is the $mer predictions, but I'm more interested in doing this right. Any reference on this subject would be greatly appreciated. Edit: I notice a similar dynamic comparing the fitted values for a random effects model with those from the OLS model with the same predictors. I guess this means the difference in$R^2$is from the incorporation of random effects into the predictions. I think this is the difference between "marginal" and "conditional"$R^2\$, but I'm still unsure of which is more honest to report.

• This question is unclear at least for Cross Validated since it does not even mention which language (R?) and package you are referring to. Mar 20, 2016 at 7:53
• Thank you @Xi'an, I have clarified that this is in R. I previously thought the code and the tags (two of which are R packages, which gamm4 is sort of a combination of) were sufficient. Mar 20, 2016 at 14:31