10
$\begingroup$

For example, if the response variable is binomially distributed, the canonical link function is the logit link. However, other link functions are available such as probit link and log-log link (but these are not canonical link functions). Canonical link functions make certain aspects of statistical inference much easier. However, sometimes canonical link functions don't suit the data well and we are forced to use a non-canonical link function to make a better regression model.

I was wondering if there are any classic situations in statistical modelling where we know right away that a canonical link function will not be suitable?

I tried to do some digging around and thought that perhaps some models like mixture models or gamma-hurdel models would involve a non-canonical link function due to its unique set up. But are there any well-known situations (e.g. bimodal response, many zeros in response) in which we know right away that a GLM with a canonical link is not a good idea?

$\endgroup$
1

1 Answer 1

8
$\begingroup$

This is a good question, though I think you need to approach this from a different angle: the link function does not work directly on the data. Recall that it converts between the linear predictor $\eta$ (your parameter estimates) and the predicted response $\mu$ -- that's it. Whether your model is appropriate for your data also depends on the underlying distribution, and thereby likelihood & variance functions chosen.

The important implication here is that different link functions give rise to different interpretations of the parameter estimates. The binomial GLM, where your response is expected as a probability $p=k/n$ with frequency weight $n$ is perhaps a very flexible example of how the link affects these parameter estimates:

  • the canonical logit link makes these log-odds, which can be back-transformed into odds ratios.
  • under a log link they become log-probabilities, which can be back-transformed into risk ratios.
  • using an identity link models the probabilities directly, which (through identity back-transform) give rise to risk differences.
  • under a complementary log-log link the model becomes akin to a discrete-time proportional hazards model, see e.g. here.

You need to remember that in all of these cases the observed data would still be probabilities or individual 0-1 responses: the likelihood remains on the probability parameter $p$ and the variance function is fixed at $p\times(1-p)$.

A particular choice of link could result in technical complications for some models because it co-determines what $\eta$ and thereby $\mu$ might be calculated during the -- usually iterative -- fitting. The logit is quite safe for the binomial case because it transforms any real & finite $\eta$ onto $]0,1[$, whereas the log is only valid for $\eta\le0$ (otherwise you would get $\mu>1$) and the identity link will very easily result in predictions that are outside of the likelihood's domain: you need $0\le\eta\le1$. Still, all of this happens for the parameter estimates and not the data.

From this I hope it's clear that the data itself should contribute relatively little to the choice of link: you should really ask yourself first what it is you want to estimate.

You mention a few cases like a hurdle Gamma model, bimodal response, or binomial GLM with many zeroes. There is no a priori reason why these could or couldn't work under their canonical link. Gamma regression has a minus-reciprocal canonical link, but many prefer the log link for interpretability. For the hurdle/zero-inflated case you would usually include a second model that handles the hurdle or zero-inflation separately (very commonly again a binomial model with any choice of link), and for bimodal response you really only care that the conditional distribution is captured well enough, specifically, that your model includes the relevant predictors of this response (again has nothing to do with the link).

$\endgroup$
1
  • 7
    $\begingroup$ Wonderful answer. I shy away from probit models because we have no interpretation of effects in that model like we do with logit. $\endgroup$ Commented Oct 12 at 12:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.