2
$\begingroup$

The reciprocal link function in exponential regression doesn't constrain to positive values, whilst on the other hand it does thoroughly space out points close to zero when fitting the linear predictor.

(a) Regarding the lack of constraint of results to non-negative predictions, my thinking is that, as long as we choose an initial trial parameter vector that results in a non-negative linear predictor, the chances of ending up with negative predictions is negligible since the sheer number of points in the neighbourhood of zero will make the predictions very accurate in that region.

(b) I am not clear as to how transforming values close to zero to be spread out over the range of the reciprocal function makes the regression more successful. I can see that it reduces leverage compared with an ordinary regression to fit the means, but is there something deeper connected with it being easier to fit a linear function to points that are reasonably far apart?

I was wondering how that reasoning sat with more seasoned modellers of exponential regression, and keen to get some more understanding around what makes a good link function over and above that it constrains predictions to appropriate values.

Thanks, Marko

$\endgroup$
1
$\begingroup$

The canonical link function is not chosen because it's logical (or "successful" whatever that means in this case) for any particular purpose (though in many cases it works out that way) -- it relates to a specific feature of the way the mean parameter appears in the density function.

If a link function doesn't suit what you do, you shouldn't use it.

In particular, if $g(\mu|\mathbf{x})$ is not linear in $\mathbf{x}$ ... you have the wrong link.

The reciprocal link function in exponential regression doesn't constrain to positive values,

It does within the range of the data but can go outside it if you extrapolate.

If it's important to avoid, you might consider whether a log-link is a better choice.

(a) Regarding the lack of constraint of results to non-negative predictions, my thinking is that, as long as we choose an initial trial parameter vector that results in a non-negative linear predictor, the chances of ending up with negative predictions is negligible since the sheer number of points in the neighbourhood of zero will make the predictions very accurate in that region.

Yes, in practice, it usually isn't a problem.

(b) I am not clear as to how transforming values close to zero to be spread out over the range of the reciprocal function makes the regression more successful. I can see that it reduces leverage compared with an ordinary regression to fit the means, but is there something deeper connected with it being easier to fit a linear function to points that are reasonably far apart?

keen to get some more understanding around what makes a good link function over and above that it constrains predictions to appropriate values.

Well, usually the main criterion is that the linear predictor is linear; if you have only factors for predictors that's obviously not a direct concern (I'd think more about things like interactions -- a good choice of link may make the model additive in the main effects which has some advantages).

However, getting feasible predictions in some part of the range may be important. In some cases this indicates a problem with the model (you may in fact know that the $E(Y|\mathbf{x}^{(0)})=0$ at some special value ofr the set of predictors, $\mathbf{x}^{(0)}$ for example, which suggests you should incorporate that knowledge in the model. In other cases you want it to produce only positive predictions for any possible set of $x$'s in which case you might look at different links (or indeed to some other way of imposing the constraint).

$\endgroup$
  • $\begingroup$ Thanks very much. This is exactly the type of detail I was hoping for. I do appreciate the point that 1/x is the canonical function and so probably has a bit of a head start on other choices. Would you know a book that condenses some of the more practical experiences of wrestling to find a link function that provides a good fit for different datasets? I'd fancy getting a better appreciation of the nuances, as it helps to extend modelling into new areas. $\endgroup$ – Marko Jul 27 '16 at 21:42
  • 1
    $\begingroup$ @Marko I don't have a book to recommend on that, though I seem to recall seeing some papers on choosing a link. You might consider asking a new question with the [references] tag (as well as the obvious ones for question). $\endgroup$ – Glen_b Jul 27 '16 at 22:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.