Distribution for random effects in Negative Binomial GLMMs

Question

I am using glmer.nb() in R for estimation a negative binomial mixed-effects model. I know in glmer() the package assumes Normal distribution with mean 0 and unknown variance. Is it the same in glmer.nb()? I have seen some papers that mention Normal distribution and some other that mention gamma function with mean 1 for random effects (which sounds a little weird to me).

Answer 1

A negative binomial distribution $\text{NegBin}(\mu, \kappa)$ with mean rate $\mu>0$ and dispersion parameter $\kappa>0$ is actually itself a GLMM already. This is because you can regard it as a Poisson random effects model with a $\lambda_i \sim \text{Gamma}(1/\kappa, 1/\kappa)$ random experimental unit effect (with mean 1) so that $Y_i | \lambda_i \sim \text{Poisson}(\lambda_i \mu)$. Of course, you can get quite similar behavior by having a normally distributed random effect $\log \lambda_i \sim N(0, \sigma)$ (the log-normal and gamma distributions are not quite identical in behavior, but somewhat similar). The main reason for why the gamma version is so popular is that the latent random effects $\lambda_i$ can just be integrated out so that the likelihood can just be written in terms of $\mu$ and $\kappa$, which does not work with a normally distributed random effect.

Once you start adding additional random effects, I really do not think it matters which of the two distributions you pick. That is, unless you were to know the true underlying distribution of random effects, but quite honestly those are usually used to approximate some unexplained variability across units in a sufficiently flexible way - i.e. the specified model is almost certainly wrong when we truly come down to it, but is hopefully a useful approximation (to paraphrase the famous quote). The normally distributed random effect approach seems to be implemented in a lot of different software packages, while I suppose a gamma random effect might admit some for of Gibbs sampling, if you are being Bayesian.