There are problems with the MASS library's glmmPQL
. It does not return a log-likelihood, so model selection will be difficult.
As for what to look at: check for warnings()
, look at fixed effect estimates and significance, look at random effects variance estimates (and confidence intervals for those if you can), and look at the overdispersion parameter estimate. I would say to also look at the log-likelihood so you can compare nested models, but... I've already said that could be an issue.
There are, however, a few other options for you.
Older versions of lme4
had a method
parameter for lmer
and glmer
which allowed you to specify PQL. While the error reporting was sometimes scant (you needed to check warnings about separability to see if your variance estimates were nonsense). However, apart from that issue the code was better in that it returned a log-likelihood which you could use.
You could also try glmer
's Laplace or quadrature (say, for 2-3 points) methods. Those are typically slower than PQL, but they are similar in that they approximate the integrand and may be more useful than an old version of glmer
or not getting a log-likelihood back from glmmPQL
.