Generalized Linear Mixed Models: Diagnostics

Question

I have a random intercept logistic regression (due to repeated measurements) and I would like to do some diagnostics, specifically concerning outliers and influential observations.

I looked at residuals to see if there are observations that stand out. But I would also like to look at something like Cook's distance or DFFITS. Hosmer and Lemeshow (2000) say that due to the lack of model diagnostic tools for correlated data, one should just fit a regular logistic regression model ignoring the correlation and use the diagnostics tools available for regular logistic regression. They argue that this would be better than doing no diagnostics at all.

The book is from 2000 and I wonder if there are methods available now for model diagnostics with mixed effects logistic regression? What would be a good approach to check for outliers?

Edit (Nov 5, 2013):

Due to the lack of responses, I am wondering if doing diagnostics with mixed models is not done in general or rather not an important step when modeling data. So let me rephrase my question: What do you do once you found a "good" regression model?

Answer 1

The diagnostic methods are indeed different for generalized linear mixed models. A reasonable one that I have seen that is based on residuals from a GLMM is due to Pan and Lin (2005, DOI: 10.1111/j.1541-0420.2005.00365.x). They have been using cumulative sums of residuals where the ordering is imposed either by the explanatory variables or by the linear predictor, thus testing either the specification of the functional form of a given predictor or the link function as a whole. The null distributions are based on simulations from the design space from the null distribution of correct specifications, and they demonstrated decent size and power properties of this test. They did not discuss outliers specifically, but I can imagine that outliers should probably throw off at least the link function by curving it too much towards the influential observation.

Answer 2

There are a lot of different opinions on what the best way to look at diagnostics for mixed models is. Generally, you will want to look at both the residuals and the standard aspects that would be examined for a non-repeated-measures model.

In addition to those, typically, you will also want to look at the random effects themselves. Methods often involve plotting the random effects by various covariates and looking for non-normality in the random effects distribution. There are many more methods (some mentioned in the prior comments), but this is usually a good start.