Say I'm doing a negative binomial regression and I'm trying to fit my model via gradient descent.

enter image description here

Given that the Poisson and Negative Binomial have the same inverse link, how might I account for the dispersion?

P.s I need to implement this via GD as I'm working with encrypted ML; I can't do inverses or transposes easily within my problem setup

  • 1
    $\begingroup$ Stata docs have a likelihood function stata.com/manuals13/rnbreg.pdf (pg 11) -- although not 100% sure if the NB2 variant of the dispersion term is the same as your screenshot. Note for max-lik people typically fix the dispersion term (or do a grid search). I've just feed it in as a parameter for back-propagation though in the past and have had fine results. $\endgroup$
    – Andy W
    Commented Nov 12, 2021 at 22:14
  • $\begingroup$ That's super cool! Mind converting this into a full answer? I'm more than happy to accept it $\endgroup$
    – IanQ
    Commented Nov 12, 2021 at 22:50

1 Answer 1


I like looking at the Stata docs when I am implementing these models loss functions in other guises. So here on pg 11 Stata has a likelihood function for their version of negative binomial regression.

ll for nbreg

Where $\alpha$ is the dispersion estimate and $u_j$ is the mean estimate (same as for Poisson regression). This is a variant called the NB2 distribution. I have an example on my blog of using this as a loss function in a pytorch deep learning model. So here is the code for torch tensors, but should be easily translatable to other coding languages:

# pytorch loss function
def nb2_loss(actual, log_pred, disp):
    m = 1/disp.exp()
    mu = log_pred.exp()
    p = 1/(1 + disp.exp()*mu)
    nll = torch.lgamma(m + actual) - torch.lgamma(actual+1) - torch.lgamma(m)
    nll += m*torch.log(p) + actual*torch.log(1-p)
    return -nll.mean()

Just a few notes -- I paramaterize $\log(\alpha)$ as disp in this loss function. So this constrains the $\alpha$ parameter to always be positive. Like all models that use backpropogation, you need decent starting parameters. I think a starting parameter of somewhere between 0 and 1 works well for the problems I have dealt with.

For a second note, I have had terrible time with using pytorch's stochastic gradient descent in all my experiments with this (even with Poisson regression and fake data so I know good starting points). So at this point I always just default to the Adam optimizer (but again good starting points for all parameters are important).

  • 1
    $\begingroup$ I tried doing this but get some unfortunate (but understandable) learning dynamics. The initial regression coefficient estimates are bad, so alpha gets small (~= high noise variance), then eventually the regression coefficients get better which drives alpha higher (~= low noise variance). So your "good starting point" comment is very true. $\endgroup$
    – daknowles
    Commented May 13, 2023 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.