Why is the canonical link of a GLM with Gamma distribution the reciprocal?

Question

I'm fitting a generalized linear model to a theoretically gamma-distributed dataset, and I'm confused about the canonical link. The gamma distribution has PDF $$ f(y;a,\lambda) = \frac{\lambda^a e^{-\lambda y}y^{a-1}}{\Gamma(a)} $$ and can be shown to belong to the exponential family: $$ f = exp\{alog(\lambda)-\lambda y+(a-1)log(y)-log(\Gamma(a))\} $$ Or, with some small changes, $$ f = exp\{\frac{y\cdot\frac{\lambda}{a}-log(\lambda)}{(-1/a)} + [(a-1)log(y)-log(\Gamma(a))]\} $$ By this representation, if the canonical link is used for a generalized linear model, $\eta = \frac{\lambda}{a} = \mathbf{x}^T\beta$. This is contrary to the canonical links in books (Introduction to Linear Regression Analysis by Montgomery, Peck, and Vining) of $\eta = 1/\lambda$. What is the problem here? Is it because of the alternative PDF of the Gamma distribution, such as in @GordonSmyth's answer in https://stats.stackexchange.com/a/474351/411578?

Answer 1

Your formulation of the Gamma distribution is:

$$ f(y;a,\lambda) = \frac{\lambda^a e^{-\lambda y} y^{a-1}}{\Gamma(a)} $$ where $a$ (often referred to as $k$ in some texts) is the shape parameter, and $\lambda$ is the rate parameter.

To express this distribution in the exponential family form, we write it as:

$$ f(y; \theta, \phi) = \exp\left(\frac{y\theta - b(\theta)}{a(\phi)} + c(y,\phi)\right) $$ For the Gamma distribution:

• $\theta = -\lambda$
• $b(\theta) = -\log(-\theta) = -\log(\lambda)$
• $a(\phi) = \frac{1}{k}$ (using $k$ instead of $a$ for clarity, where $k$ is the shape)
• $c(y, \phi) = (k-1)\log y - \log \Gamma(k)$

In this exponential family form:

$$ \theta = -\lambda, \quad b(\theta) = -\log(-\theta) $$ In the context of exponential families, $b(\theta)$ is known as the cumulant generating function. The derivative $b'(\theta)$ provides the expected value (or mean) of the distribution. This relationship is important for linking the natural parameter $\theta$ to the mean $\mu$, which is often the parameter of interest in statistical modeling.

Thus, the mean of the distribution $\mu$ can be obtained from the derivative of $b(\theta)$:

$$ b'(\theta) = \frac{d}{d\theta}(-\log(-\theta)) = \frac{d}{d\theta}(-\log(\lambda)) = -\frac{1}{\lambda} $$ Since $\theta = -\lambda$, and $\mu = -\frac{1}{\theta} = 1/\lambda$, we see that:

$$ \mu = 1/\lambda $$ The canonical link function $g(\mu)$ is defined as the function that equates the linear predictor $\eta$ to the natural parameter $\theta$, i.e., $\eta = g(\mu) = \theta$. For the Gamma distribution, since $\theta = -\lambda$ and $\mu = 1/\lambda$, the canonical link function is the reciprocal link:

$$ g(\mu) = -1/\mu $$ The confusion in your question arises from the parameters used and their interpretations. When you equate $\eta = \frac{\lambda}{a}$, it seems like a mix-up in the interpretation of parameters in your formulation. The canonical link function should straightforwardly be $\eta = -1/\mu$, which implies $\eta = -\lambda$, fitting with the notion that for the Gamma distribution with rate $\lambda$, the reciprocal link $\eta = 1/\mu$ is natural and correct. This aligns with $\mu = 1/\lambda$, thus $\eta = -1/\mu$ makes $\eta = -\lambda$, the correct canonical form for $\theta$.