# Negative Binomial Regression Dispersion Parameter

I have seen conflicting definitions in the literature as to whether the variance of the response under a negative negative binomial regression model is specified as $\mu + \kappa \mu^2$ or as $\mu + \dfrac{\mu^2}{\kappa}$, where $\mu$ is the mean. Which one does R take it to be? Because when I fit the model I get a "theta", as R calls it, of 1.5. It seems more rational that it should be the latter, but I would like to make sure. Thanks in advance!

The model specification for the negative binomial GLM as encoded in glm.nb in the MASS library is:

$$\text{Var}(Y) = \mu + {\mu^2 \over \theta}$$

(from p. 206 of Venables and Ripley, MASS). Note that this is different from the (less common) specification in the question.

Other libraries, however, may use the alternative formulation:

$$\text{Var}(Y) = \mu + k\mu^2$$

In your case, since the data structure returned contains a parameter explicitly labelled "theta", I assume you are using glm.nb, in which case the first formulation is the correct one.

• The formulation $\mathrm{Var}(y) = \mu + \alpha \cdot \mu = \phi \cdot \mu$ is not "incorrect" but simply a different negative binomial model, typically called the "NB1" parameterization. In contrast, the MASS package uses the "NB2" parameterization (which is used more frequently in the literature, especially the statistical literature). – Achim Zeileis Apr 15 '18 at 21:25
• @AchimZeileis - Good observation (+1) and I've edited the answer as a result. Thanks! – jbowman Apr 15 '18 at 22:03
• Apologies, I was in fact going for the NB2 model rather than that NB1 model- it was a simple typo which I have now corrected. Thanks! – shrut9 Apr 20 '18 at 10:18