Calculation of canonical link function in GLM

Question

I thought that the canonical link function $g(\cdot)$ comes from the natural parameter of exponential family. Say, consider the family $$ f(y,\theta,\psi)=\exp\left\{\frac{y\theta-b(\theta)}{a(\psi)}-c(y,\psi)\right\} $$ then $\theta=\theta(\mu)$ is the canonical link function. Take Bernoulli distribution as an example, we have $$ P(Y=y)=\mu^{y}(1-\mu)^{1-y}=\exp\left\{y\log\frac{\mu}{1-\mu}+\log{(1-\mu)}\right\} $$ So, the canonical link function $$g(\mu)=\log\frac{\mu}{1-\mu}$$

But when I see this slide, it claims that $$ g'(\mu)=\frac{1}{V(\mu)} $$ Though it can be easily verified for this particular distribution (and some other distributions, like Poisson distribution), I can't see the equivalence for the general case. Can anyone give hints? Thank you~

Answer 1

The variance function for the Bernoulli variable is $V(\mu) = \mu(1-\mu)$. We easily check that with the canonical link $g(\mu) = \log \frac{\mu}{1-\mu} = \log \mu - \log(1-\mu)$ then $$g'(\mu) = \frac{1}{\mu} + \frac{1}{1-\mu} = \frac{1 - \mu + \mu}{\mu(1-\mu)} = \frac{1}{\mu(1-\mu)} = \frac{1}{V(\mu)}.$$

For the general case one derives from the definition that $$E(Y) = \mu = b'(\theta) \quad \text{ and } \quad \text{Var}(Y) = b''(\theta) a(\psi),$$ see e.g. page 28-29 in McCullagh and Nelder. With $g$ the canonical link we have $\theta = g(\mu) = g(b'(\theta))$, and the variance function is defined as $b''(\theta)$, which in terms of $\mu$ becomes $$V(\mu) = b''(g(\mu)).$$ By differentiation of the identity $\theta = g(b'(\theta))$ we get $$1 = g'(b'(\theta)) b''(\theta) = g'(\mu) V(\mu),$$ which gives the general relation between the canonical link function and the variance function.

In the construction of quasi-likelihood functions it is natural to start with the relation between the mean and the variance, given in terms of the variance function $V$. In this context the anti-derivative of $V(\mu)^{-1}$ can be interpreted as a generalization of the link function, see, for instance, the definition of the (log) quasi-likelihood on page 325 (formula 9.3) in McCullagh and Nelder.