84
$\begingroup$

The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in non-negative data, in a way that log-transformed OLS can't do without either WLS or some sort of heteroskedasticity-consistent VCV estimator.

I would use it more for routine non-negative data modeling, but I don't know anyone else that uses it, I haven't learned it in a formal classroom setting, and the literature that I read never uses it. Whenever I Google something like "practical uses of gamma GLM", I come up with advice to use it for waiting times between Poisson events. OK. But that seems restrictive and can't be its only use.

Naively, it seems like the gamma GLM is a relatively assumption-light means of modeling non-negative data, given gamma's flexibility. Of course you need to check Q-Q plots and residual plots like any model. But are there any serious drawbacks that I am missing? Beyond communication to people who "just run OLS"?

$\endgroup$
54
$\begingroup$

The gamma has a property shared by the lognormal; namely that when the shape parameter is held constant while the scale parameter is varied (as is usually done when using either for models), the variance is proportional to mean-squared (constant coefficient of variation).

Something approximate to this occurs fairly often with financial data, or indeed, with many other kinds of data.

As a result it's often suitable for data that are continuous, positive, right-skew and where variance is near-constant on the log-scale, though there are a number of other well-known (and often fairly readily available) choices with those properties.

Further, it's common to fit a log-link with the gamma GLM (it's relatively more rare to use the natural link). What makes it slightly different from fitting a normal linear model to the logs of the data is that on the log scale the gamma is left skew to varying degrees while the normal (the log of a lognormal) is symmetric. This makes it (the gamma) useful in a variety of situations.

I've seen practical uses for gamma GLMs discussed (with real data examples) in (off the top of my head) de Jong & Heller and Frees as well as numerous papers; I've also seen applications in other areas. Oh, and if I remember right, Venables and Ripley's MASS uses it on school absenteeism (the quine data; Edit: turns out it's actually in Statistics Complements to MASS, see p11, the 14th page of the pdf, it has a log link but there's a small shift of the DV). Uh, and McCullagh and Nelder did a blood clotting example, though one or both of those may have been natural link.

Then there's Faraway's book where he did a car insurance example and a semiconductor manufacturing data example.

There are some advantages and some disadvantages to choosing either of the two options. Since these days both are easy to fit; it's generally a matter of choosing what's most suitable.

It's far from the only option; for example, there's also inverse Gaussian GLMs, which are more skew/heavier tailed (and even more heteroskedastic) than either gamma or lognormal.

As for drawbacks, it's harder to do prediction intervals. Some diagnostic displays are harder to interpret. Computing expectations on the scale of the linear predictor (generally the log-scale) is harder than for the equivalent lognormal model. Hypothesis tests and intervals are generally asymptotic. These are often relatively minor issues.

It has some advantages over log-link lognormal regression (taking logs and fitting an ordinary linear regression model); one is that mean prediction is easy.

$\endgroup$
  • 3
    $\begingroup$ Should be it "Gamma" or "gamma"? We know it's not named for a person. I've seen lower case "g" much more frequently. Clearly the distribution is named for the function, which goes back to the 18th century. $\endgroup$ – Nick Cox Aug 16 '13 at 9:43
  • 2
    $\begingroup$ The $\Gamma$ notation is the only reason I've seen for that use. With distributions generally, upper case usually echoes surnames, e.g. Poisson or Gaussian, as you know. $\endgroup$ – Nick Cox Aug 16 '13 at 10:31
  • $\begingroup$ @NickCox I have changed it as you suggest, and I fixed "Inverse Gaussian" while I was at it. $\endgroup$ – Glen_b Aug 16 '13 at 10:56
  • 1
    $\begingroup$ @Gleb_b: Do you still use the log link with the inverse Gaussian family? $\endgroup$ – Dimitriy V. Masterov Aug 19 '13 at 0:54
  • $\begingroup$ @DimitriyV.Masterov It's less used so it's harder to generalize. From what I've seen, it's pretty common to use a log-link with inverse Gaussian, but other links may be suitable in some situations, such as an inverse link. $\endgroup$ – Glen_b Aug 19 '13 at 1:02
28
$\begingroup$

That's a good question. In fact, why don't people use generalised linear models (GLM) more is also a good question.

Warning note: Some people use GLM for general linear model, not what is in mind here.

  • It does depend where you look. For example, gamma distributions have been popular in several of the environmental sciences for some decades and so modelling with predictor variables too is a natural extension. There are many examples in hydrology and geomorphology, to name some fields in which I have strayed.

  • It is hard to pin down quite when to use it beyond an empty answer of whenever it works best. Given skewed positive data I will often find myself trying gamma and lognormal models (in GLM context log link, normal or Gaussian family) and choosing which works better.

  • Gamma modelling remained quite difficult to do until fairly recently, certainly as compared with say taking logs and applying linear regressions, without writing a lot of code yourself. Even now, I'd guess that it isn't equally easy across all the major statistical software environments.

  • In explaining what is used and what is not used, despite merits and demerits, I think you always come down to precisely the kind of factors you identify: what is taught, what is in the literature that people read, what people hear talked about at work and at conferences. So, you need a kind of amateur sociology of science to explain. Most people seem to follow straight and narrow paths within their own fields. Loosely, the larger the internal literature in any field on modelling techniques, the less inclined people in that field seem to be to try something different.

$\endgroup$
  • 1
    $\begingroup$ How do you determine which works better? $\endgroup$ – Dimitriy V. Masterov Aug 19 '13 at 0:56
  • 7
    $\begingroup$ I look at likelihoods, R-squares (despite what people say), confidence intervals around parameter estimates, plots of observed vs fitted, residual vs fitted, etc. If there were science favouring one model over another, that would weigh too, but in my experience the science is not so well formed. How else could it be done? $\endgroup$ – Nick Cox Aug 19 '13 at 7:35
  • $\begingroup$ @NickCox What should we look out for when analysis observed vs fitted, residuals vs fitted and normal qq plot? I understand this might differ between models. Could you give an example for gamma, poisson and negative binomial? Thanks $\endgroup$ – tatami Oct 10 '17 at 8:20
  • $\begingroup$ @tatami That's an entire new question, or more, I think. If you ask it, you'll see who bites. I've not ever thought that a gamma model and a negative binomial model were rivals in any project, but that could be failure of imagination or experience. $\endgroup$ – Nick Cox Oct 10 '17 at 8:41
13
$\begingroup$

Gamma regression is in the GLM and so you can get many useful quantities for diagnostic purposes, such as deviance residuals, leverages, Cook's distance, and so on. They are perhaps not as nice as the corresponding quantities for log-transformed data.

One thing that gamma regression avoids compared to the lognormal is transformation bias. Jensen's inequality implies that the predictions from lognormal regression will be systematically biased because it's modeling transformed data rather than the transformed expected value.

Also, gamma regression (or other models for nonnegative data) can cope with a broader array of data than the lognormal due to the fact that it can have a mode at 0, such as you have with the exponential distribution, which is in the gamma family, which is impossible for the lognormal.

I have read suggestions that using the Poisson likelihood as a quasi-likelihood is more stable. They're conjugates of each other. The quasi-Poisson also has the substantial benefit of being able to cope with exact 0 values, which trouble both the gamma and, especially, the lognormal.

$\endgroup$
11
$\begingroup$

In my opinion, it assumes that the errors lie on a family of gamma distributions, with the same shapes, and with the scales changing according the related formula.

But it is difficult to do model diagnosis. Note that the simple QQ plot is not suitable here, because it is about the same distribution, while ours is a family of distributions with different variances.

Naively, the residuals plot can be used to see that they have different scales but the same shape, usually with long tails.

In my experience, the gamma GLM may be tried for some long tail distributed problems, and it is widely used in insurance and environment sectors, etc. But the assumptions are difficult to test, and the model does not perform well usually, so different papers argue to use other family distributions with the same problem, like inverse Gaussian, etc. In practice, it seems that such choices depends on expert judgement with the industrial experience. This limits the use of the gamma GLM.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.