I fitted a mixed logit model with crossed random effects in lme4_1.1-21::glmer to some experimental binary data. The maximal random-effect structure justified by the design was simplified due to singularity problems.

Here is the final model:

mod_3 <- glmer(y ~ A*B +  (A*B| Subj) + (0 + B|Itm) + (0 + A:B|Itm), 
        glmerControl(optimizer='bobyqa', optCtrl = list(maxfun=400000)))

y is the binary outcome variable. A and B are within-subjects within-items categorical predictors, deviation-contrast coded with values -.5/.5. 16 obs per Subj and 176 obs per Itm.

Excess zeros in the data?

The overall probability of a positive response Pr(y = 1) is 0.115 in the data and 39% of subjects never produced a positive response which made me worry the data might have issues of zero-inflation.

I examined the residuals of the model for zero-inflation. However, the two diagnostic approaches I used lead me to opposite conclusions and I'd appreciate any steer on the matter.

Residual diagnostic - Approach 1

I followed Gelman and Hill (2016, p.97 ) and plotted the binned residuals. Most binned residuals fall outside the +/-2 SE boundaries, and the diagonal dense line of points outside the boundaries suggests the model struggles with fitted values below 0.05, over-predicting positive responses.

I took this as a sign of the excess number of zeros in the dataset.

  x = predict(mod_3, type = 'response'),
  y = resid(mod_3, type='response'), 
  xlab = "Expected Pr (y = 1)",
  cex.pts=.5, col.int="black", 
  main="Binned residual (response) plot")


Residual diagnostic - Approach 2

I then discovered the DHARMa package and used it to produce residual diagnostic plots for the model.

Here, however, there doesn't seem to be any pattern in the residuals; no sign of zero-inflation or excess variance.

simulationOutput_mod_3 <- simulateResiduals(fittedModel = mod_3)


The test for zero-inflation is also not-significant:


ratioObsSim = 0.9973, p-value = 0.816


Things look "massier" when looking at residuals grouped by subjects, but again the test for zero-inflation is not significant (ratioObsSim = 0.85513, p-value = 0.24)



I'm the developer of the DHARMa package. It's hard to say what's going on without seeing your data, but I suspect that the issue arises as follows:

  • As one can see in the DHARMa plots, your fixed effects design just creates 4 predictions (I assume both A,B only have 2 levels, so you have 4 values from A*B)

  • In your binned plots, however, there are a lot more values on the x-axis, which suggests to me that you use the fixed effects + REs for creating the predictions for the binned plots. This, however, creates spurious correlations, as demonstrated here https://github.com/florianhartig/DHARMa/issues/43, which is why DHARMa only uses fixed effects for calculating the res ~ pred plots (leading to only 4 predicted values in your first DHARMa plot).

My best guess is that your model is fine, and the pattern in the binned plots is simply an artefact created by erroneously including the REs in the prediction for the binning.

p.s.: if you have questions regarding DHARMa, you will get a faster answer if you post them here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.