3
$\begingroup$

For unbounded continuous responses, Gaussian errors are the analyst's default model for many reasons, one of them being that their ML estimate coincides with the OLS estimate that has many desirable properties.

I am not aware of similar results for strictly positive continuous response variables, and I doubt they exist. Previous answers (and my own knowledge) seem to punt on this question, as in https://stats.stackexchange.com/a/79517/36229 and https://stats.stackexchange.com/a/136850/36229.

In the second answer, for instance, it says that in the gamma model the variance is proportional to the squared mean, and to the cubed mean in the inverse Gaussian model. And if I'm not mistaken, the lognormal model also has the variance proportional to the squared mean, with the additional interpretive benefit of being the logarithm of a Gaussian.

In the absence of a theoretical basis for one or the other, what should be my go-to model, and why? Or, as per whuber's suggestion in the comments, is there a principled way to select one or the other?

Or, is the difference small in practice and the one I choose is a matter of convenience and interpretive relevance (as with logit vs probit)?

And what about link functions? The canonical gamma link is the inverse mean, but I usually see the log mean used in practice. For argument's sake, let's assume we're using a log link. As per the comments, the link function is probably more important than the error distribution itself, but it's also a bigger/different question. Not to mention the fact that log links are used with all of the gamma, lognormal, and inverse Gaussian distributions. And inverse links are tough to wrap your head around.

Edit: for what it's worth, I found some guidance here and here that use simulations, and recommend the gamma distribution overall. Glen_b demonstrates that the log link tends to over-correct gamma errors, inducing left skew rather than symmetry but, like many others, seems equivocal overall on which one to use when. Firth (1988) shows that the gamma is slightly more efficient under certain kinds of misspecification.

Edit 2: You could always fit both and choose one based on some goodness-of-fit criterion, but in my opinion that's neither principled nor satisfying.

$\endgroup$
  • 5
    $\begingroup$ Very interesting question: from my perspective it's backward. I'd say for strictly positive responses (more generally non-negative), the best default link is logarithmic. The question of error family is secondary. We've had a century of over-emphasis on Gaussian errors for linear models, which is naturally not Gauss's fault. For one provocative statement, see blog.stata.com/tag/poisson-regression (I think we should re-discover a slightly unfashionable term and talk about log-linear models. Being Poisson is arguably not the most important fact about Poisson regression.) $\endgroup$ – Nick Cox Feb 20 '15 at 19:12
  • 2
    $\begingroup$ This question is close to the one yesterday at stats.stackexchange.com/questions/138383. Glen_b's answer there would go a long way towards addressing this question, too. The key is to pay attention to the error structure as well as to the possible form of the relationship (the "link" function). I would add that the premise you should have a standard default ("go-to") model to try in all circumstances might be overly limiting and counterproductive. Why not instead adopt a standard set of procedures to help select and identify appropriate models, without favoring any particular one? $\endgroup$ – whuber Feb 20 '15 at 19:40
  • 2
    $\begingroup$ I agree with @NickCox. You can read an answer of mine in favor of log-transforming the left-hand-side (and positive right-hand-side) variables here: stats.stackexchange.com/questions/107610/… $\endgroup$ – Bill Feb 20 '15 at 19:44
  • 2
    $\begingroup$ I actually show that the log-transformation makes Gammas left skew; the log-link in a GLM is not a transformation. Taking logs when the data are gamma and fitting using OLS introduces a bias (removable, at least approximately) and is asymptotically less efficient. That wouldn't be a warning against using the gamma but against treating a gamma as if it were lognormal, though if you know it's gamma (or approximately gamma), the bias issue isn't necessarily such a big deal. My own approach is to look at log-response -- if the logs look left skew, but the variance is constant, try a gamma. $\endgroup$ – Glen_b Oct 29 '15 at 17:44
  • 1
    $\begingroup$ I've expanded a little $\endgroup$ – Glen_b Oct 29 '15 at 23:04
3
$\begingroup$

I really don't think there should be a "default" distribution here (and it's not typically the most critical part). In some particular applications one might be able to argue for a default, but generally I think the approach isn't likely to be fruitful.

In many cases the choice may depend on a host of issues. Let's assume constant spread on the log scale for now ... if I am trying to predict the conditional mean of the data then (where both seem otherwise suitable) I may lean toward gamma GLM with log link over lognormal regression but if I want a prediction interval I may lean toward lognormal.

If I recall correctly, what I showed is that the log-transformation makes Gammas left skew; the log-link in a GLM is not a transformation, and there's no issue with the log link.

So taking logs when the data are gamma and fitting using OLS introduces a bias and is less efficient. That wouldn't be a warning against using the gamma but against treating a gamma as if it were lognormal*, though if you know it's gamma (or approximately gamma), the bias issue isn't necessarily such a big deal since you can fairly readily adjust for that.

* at least in the sense that the procedure is efficient at the lognormal

My own approach is to look at the (conditional)** log-response -- if the variance is constant but the distribution looks left skew, try a gamma:

![enter image description here

** or the residuals from a model on the log response

But you might consider other things; perhaps a Weibull, for example (modelling the scale parameter, as in Weibull survival models, is also variance-proportional-to-mean-squared and I've used it successfully on data I might have used a gamma on); on the log scale it can look distinctly left skew or (for low values of the shape parameter) nearly symmetric.

If the logs look (conditionally) symmetric, consider a lognormal (but it still may be gamma if the shape parameter is large). If they're right skew, perhaps an inverse gamma.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.