Are Canonical link functions and the link functions the same thing?

If not, can anyone tell the difference between them?

I know a link function is a function that links a linear predictor to the response variable.

Then isn't that a canonical link function too? I'm so confused, could anyone help?

Also if we're taking the distribution of the response variable, say $Y$ as a $lognormal$ distribution, i.e $Y \sim LogN(\mu,{\sigma}^2)$, then what should be the natural link function to be applied?

What I did was, first transform the distribution into $Normal$ distribution and hence use the link function $g(\mu) =\mu$, but in the solution it's given as $log(\mu)$, how is that?

Can anyone help?

  • 2
    $\begingroup$ I changed the title because if you're talking about canonical link functions you mean generalized linear models (as your chosen tag also indicates), not general linear models -- the two are different. $\endgroup$
    – Glen_b
    Sep 3 '16 at 10:48
 The canonical link is a type of link function.

Per Wikipedia: The link function provides the relationship between the linear predictor and the mean of the distribution function.... It makes sense to try to match the domain of the link function to the range of the distribution function's mean.

So there isn't one correct link function for a given distribution - it will depend on the problem.

By linear predictor of the mean of the response, the above quote means $E(Y) = {\bf X' \beta}$, where ${\bf X} = (x_1, ..., x_n)$ are the independent variables and ${\bf \beta} = (\beta_1, ..., \beta_p)$ are the coefficients. Note that the link function is $\eta = {\bf X' \beta}$. This is always true.

For example, if our response, $Y \in [0,1]$, then somehow we need to ensure that our linear predictor ${\bf X' \beta} \in [0,1]$. But since $X$, $\beta \in \mathbb{R}$, this means ${\bf X' \beta} \in \mathbb{R}$ unless we put some restrictions or apply a transformation. That's a problem because then $E(Y)$ is not guaranteed to be bound between 0 and 1.

To fix this, we can apply a transformation that takes any real number and returns a value between 0 and 1. A common transformation is the logit (or sigmoid or logistic) function:

$$ logit(x) = \frac{e^{x}}{1+e^{x}} $$

It also helps that this function monotonically increases - for inputs going from large negative to large positive numbers, the output goes from 0 to 1.

Also note that its symmetric about 0. This could be a desirable property or not. (For example, if your function decays more slowly to zero than it increases to 1, then consider using the complementary log-log link)

We call this a logit link function.

Sometimes you'll see it written in terms of the probability, $\pi_i$ of getting a 1, i.e. $P(Y_i = 1|X) = \pi_i$. In this case, we write $\eta_i = logit(\pi_i) = log(\frac{\pi_i}{1-\pi_i}) = {\bf X'\beta}$. This is the "log odds of $P(Y_i = 1|X)$".

In your question, the canonical link is $\eta = \mu$. In other words, our response can take any real value. So $E(Y) \in \mathbb{R}$. That means we don't need to restrict ${\bf X'\beta}$. So the link function basically has no impact. In other words, $\eta_i = \mu_i = {\bf X'\beta}$.

  • $\begingroup$ You seem to equate "canonical" with "correct." That's not what "canonical" means. There's a mathematical reason that Wikipedia article associates a unique, natural (i.e., canonical) link function with each of the distribution families it lists. Consequently your answer doesn't appear to address the question. $\endgroup$
    – whuber
    Oct 12 '16 at 14:26
  • $\begingroup$ I didn't mean for canonical to equate to correct. I added that to clarify that there can be many different types of links. No one correct link. As well, the OP seems to be confused with what a link function actually does. My answer is an attempt to add clarity around that. $\endgroup$
    – ilanman
    Oct 12 '16 at 14:40
  • $\begingroup$ Because the question asks about the difference between a canonical link and an arbitrary link, it is a duplicate, so I voted to close it as such. $\endgroup$
    – whuber
    Oct 12 '16 at 14:42

Not the answer you're looking for? Browse other questions tagged or ask your own question.