I have seen conflicting definitions in the literature as to whether the variance of the response under a negative negative binomial regression model is specified as $\mu + \kappa \mu^2$ or as $\mu + \dfrac{\mu^2}{\kappa}$, where $\mu$ is the mean. Which one does R take it to be? Because when I fit the model I get a "theta", as R calls it, of 1.5. It seems more rational that it should be the latter, but I would like to make sure. Thanks in advance!


The model specification for the negative binomial GLM as encoded in glm.nb in the MASS library is:

$$\text{Var}(Y) = \mu + {\mu^2 \over \theta}$$

(from p. 206 of Venables and Ripley, MASS). Note that this is different from the (less common) specification in the question.

Other libraries, however, may use the alternative formulation:

$$\text{Var}(Y) = \mu + k\mu^2$$

In your case, since the data structure returned contains a parameter explicitly labelled "theta", I assume you are using glm.nb, in which case the first formulation is the correct one.

  • 1
    $\begingroup$ The formulation $\mathrm{Var}(y) = \mu + \alpha \cdot \mu = \phi \cdot \mu$ is not "incorrect" but simply a different negative binomial model, typically called the "NB1" parameterization. In contrast, the MASS package uses the "NB2" parameterization (which is used more frequently in the literature, especially the statistical literature). $\endgroup$ – Achim Zeileis Apr 15 '18 at 21:25
  • 1
    $\begingroup$ @AchimZeileis - Good observation (+1) and I've edited the answer as a result. Thanks! $\endgroup$ – jbowman Apr 15 '18 at 22:03
  • $\begingroup$ Apologies, I was in fact going for the NB2 model rather than that NB1 model- it was a simple typo which I have now corrected. Thanks! $\endgroup$ – shrut9 Apr 20 '18 at 10:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.