I am running through the diagnostics of two logistic regressions and two equivalent GLMMs with their only differences being crossed random effects (intercepts only). The output for the diagnostics look problematic, and I was curious from others here whether or not the random model is appropriate at this point. Many of the functions that compute these values give warnings about "these may not be appropriate for GLMMs", but I'm not sure how else to piece apart how well the model is doing.
Residuals of Model
Being a bit confused on which residuals to use for logistic GLMMs and when they show issues, I have included multiple versions below. First, I ran the binned residuals of the fixed models (top of plot) and random models (bottom of plot). The fixed models are both near or above 95% within error bounds whereas the random models are 70-72% and have strange S patterns:
Most bizarre of the diagnostics I've run so far are the density of the residuals:
Next, I looked at the outliers of the model. First the fixed effects model leverage of data points, which looks very precise and has not outliers:
And the random models:
Then Cook's distance for the fixed and random models seems to indicate that the random models are fairly extreme, some distances in the thousands:
Another bizarre feature of the models is that the RMSE is actually lower in the random effects models compared to the writing models despite the large fluctuation in residuals and outliers:
The random model is too complex in my opinion and may not be able to estimate much variance the way it is specified. The random models look something like this, with all of the predictors centered and scaled with the
glmer(Y ~ (X1+X2+X3)*X4 + X5 + X6 + (1|Subject) + (1|Item) family = binomial()
Perhaps it is misspecified? Perhaps the functions I am using are truly not useful for GLMMs? Any answers would be appreciated. For the record, most of what I am using is from the
performance package, but some of this is basic R, such as the Cook's distance and density plots.