I'm trying to model a response variable y with respect to a nested variable x in R. First of all, I fitted a linear mixed model (LMM) as it follows:
m0 <- lmer(y ~ x + (x | group), data=df)
But, after looking at the residuals and the qqplot, I note that the LMM model violates the normality and homoscedastic assumptions. Also I observe that the higher the values of the fitted values, higher is the variance of residuals.
Alternatively, I try to fit a GLMM model with a Gamma family and log link:
m1 <- glmer(y ~ x + (x | group), data=df, family=Gamma(link='log'))
Now, the residuals show more homogeneity and the AIC value is considerably smaller:
AIC m0 42159,54 m1 39429,50
What are the possible steps for a model validation in this case?
A qqplot for a gamma distribution family seems to be unfeasible.
Are there any statistical tests to check the validity of the model?