0
$\begingroup$

Problem

I am running through the diagnostics of two logistic regressions and two equivalent GLMMs with their only differences being crossed random effects (intercepts only). The output for the diagnostics look problematic, and I was curious from others here whether or not the random model is appropriate at this point. Many of the functions that compute these values give warnings about "these may not be appropriate for GLMMs", but I'm not sure how else to piece apart how well the model is doing.

Residuals of Model

Being a bit confused on which residuals to use for logistic GLMMs and when they show issues, I have included multiple versions below. First, I ran the binned residuals of the fixed models (top of plot) and random models (bottom of plot). The fixed models are both near or above 95% within error bounds whereas the random models are 70-72% and have strange S patterns:

enter image description here

Most bizarre of the diagnostics I've run so far are the density of the residuals:

enter image description here

Outliers

Next, I looked at the outliers of the model. First the fixed effects model leverage of data points, which looks very precise and has not outliers:

enter image description here

And the random models:

enter image description here

Then Cook's distance for the fixed and random models seems to indicate that the random models are fairly extreme, some distances in the thousands:

enter image description here

Model Indices

Another bizarre feature of the models is that the RMSE is actually lower in the random effects models compared to the writing models despite the large fluctuation in residuals and outliers:

enter image description here

My Guess

The random model is too complex in my opinion and may not be able to estimate much variance the way it is specified. The random models look something like this, with all of the predictors centered and scaled with the scale function:

glmer(Y
~ (X1+X2+X3)*X4
+ X5
+ X6
+ (1|Subject)
+ (1|Item)
family = binomial()

Perhaps it is misspecified? Perhaps the functions I am using are truly not useful for GLMMs? Any answers would be appreciated. For the record, most of what I am using is from the performance package, but some of this is basic R, such as the Cook's distance and density plots.

$\endgroup$

1 Answer 1

0
$\begingroup$

After doing a decent amount of reading, typical residual inspection done in Gaussian regression and lmer models that handle Gaussian data is no issue, but because I am running a GLMM, the residuals have to be inspected in a different way.

Thankfully, the DHARMa package has some useful ways of approaching this, including some nice plots that are easy to obtain:

enter image description here

There are also tests of outliers that are equally easy to run:

DHARMa outlier test based on exact binomial test with
    approximate expectations

data:  model
outliers at both margin(s) = 47, observations = 6840, p-value
= 0.3408
alternative hypothesis: true probability of success is not equal to 0.007968127
95 percent confidence interval:
 0.005053048 0.009127070
sample estimates:
frequency of outliers (expected: 0.00796812749003984 ) 
                                           0.006871345 
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.