6
$\begingroup$

I was wondering why the Gamma deviance formula is given as: $$2 \sum [ -log(\frac{y_i}{\mu_i}) + \frac{y_i-\mu_i}{\mu_i} ] $$

Shouldn't the 2nd term become zero after the summation is conducted?

$\endgroup$
7
$\begingroup$

The general derivation of the deviance for a GLM family is given in Section 5.4 of Dunn and Smyth (2018) (the book that you mentioned in a previous post). You can insert the form of the gamma density to get the result, but the density has to parametrized in the right way.

A common way to write the gamma density is as $$f(y;\alpha,\beta)=\frac{y^{\alpha-1}e^{-y/\beta}}{\beta^\alpha\Gamma(\alpha)}$$ with $E(y)=\alpha\beta=\mu$ and var$(y)=\alpha\beta^2=V(\mu)\phi$ with $V(\mu)=\mu^2$ and $\phi=1/\alpha$. Converting to the log-scale gives $$\log f(y;\alpha,\beta)= -y/\beta-\alpha\log\beta+(\alpha-1)\log y-\log\Gamma(\alpha)$$ Reparametrizing to $\mu$ and $\phi$ gives $$\log f(y;\mu,\phi)= t(y,\mu)/\phi+a(y,\phi)$$ with $$t(y,\mu)=-y/\mu-\log\mu$$ and $$a(y,\phi)=(\log\phi)/\phi+(1/\phi-1)\log y -\log\Gamma(1/\phi).$$ The unit deviance is defined as $$d(y,\mu)=2\left\{t(y,y)-t(y,\mu)\right\}$$ with in this case $$t(y,y)-t(y,\mu)=-1-\log y+ y/\mu+\log\mu=(y-\mu)/\mu-\log(y/\mu).$$

Finally, the total deviance is $$D=\sum_{i=1}^n w_i d(y_i,\mu_i)$$ where the $w_i$ are the prior weights. If the prior weights are all 1, then this agrees with the deviance formula in your question.

It is true that the $(y-\mu)/\mu$ terms often do sum to zero when evaluated at the fitted values, $\mu_i=\hat\mu_i$, but not always. The GLM maximum likelihood equations solve $$\sum_{i=1}^n w_i x_{ij} \frac{y_i-\mu_i}{g'(\mu_i) V(\mu_i)}=0$$ where the $x_{ij}$ are covariate values, $g'$ is the derivative of the link function and $V(\mu)$ is the variance function. For the gamma distribution, $V(\mu)=\mu^2$. If a log-link is used, then $g'(\mu) = 1/\mu$. If a log-link is used and the covariates include an intercept term, then the likelihood equations imply $$\sum_{i=1}^n w_i \frac{y_i-\mu_i}{\mu_i}=0$$ In this case, the $(y-\mu)/\mu$ terms will not contribute to the total deviance. Nevertheless, the terms can't be ignored entirely because they are part of the unit deviances and will still contribute to the deviance residuals.

Reference

Dunn, PK, and Smyth, GK (2018). Generalized linear models with examples in R. Springer, New York, NY. https://www.amazon.com/Generalized-Linear-Examples-Springer-Statistics/dp/1441901175

$\endgroup$
8
  • $\begingroup$ yes, silly mistake $\endgroup$ – Maverick Meerkat Jun 27 '20 at 16:34
  • $\begingroup$ The reason I got confused is that I saw an application of this in a [paper][1] where the 2nd term was omitted. But I guess it's just a mistake. [1]: jstor.org/stable/2981736?seq=1 $\endgroup$ – Maverick Meerkat Jul 2 '20 at 8:52
  • $\begingroup$ @DavidRefaeli I have to correct my answer. It would be helpful if you added the reference to your question in order to provide the question with some motivation. $\endgroup$ – Gordon Smyth Jul 3 '20 at 4:54
  • $\begingroup$ yes you are totally right. Thanks for the much appreciated clarification. $\endgroup$ – Maverick Meerkat Jul 3 '20 at 6:32
  • 1
    $\begingroup$ @D1X OK, I've added a reference to the book. $\endgroup$ – Gordon Smyth Sep 3 '20 at 3:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.