When to use gamma GLMs?

Question

The gamma distribution can take on a pretty wide range of shapes, and given the link between the mean and the variance through its two parameters, it seems suited to dealing with heteroskedasticity in non-negative data, in a way that log-transformed OLS can't do without either WLS or some sort of heteroskedasticity-consistent VCV estimator.

I would use it more for routine non-negative data modeling, but I don't know anyone else that uses it, I haven't learned it in a formal classroom setting, and the literature that I read never uses it. Whenever I Google something like "practical uses of gamma GLM", I come up with advice to use it for waiting times between Poisson events. OK. But that seems restrictive and can't be its only use.

Naively, it seems like the gamma GLM is a relatively assumption-light means of modeling non-negative data, given gamma's flexibility. Of course you need to check Q-Q plots and residual plots like any model. But are there any serious drawbacks that I am missing? Beyond communication to people who "just run OLS"?

Answer 1

The gamma has a property shared by the lognormal; namely that when the shape parameter is held constant while the scale parameter is varied (as is usually done when using either for models), the variance is proportional to mean-squared (constant coefficient of variation).

Something approximate to this occurs fairly often with financial data, or indeed, with many other kinds of data.

As a result it's often suitable for data that are continuous, positive, right-skew and where variance is near-constant on the log-scale, though there are a number of other well-known (and often fairly readily available) choices with those properties.

Further, it's common to fit a log-link with the gamma GLM (it's relatively more rare to use the natural link in my experience though it makes good sense in some contexts). What makes it slightly different from fitting a normal linear model to the logs of the data is that on the log scale the gamma is left skew to varying degrees while the normal (the log of a lognormal) is symmetric. This makes it (the gamma) useful in a variety of situations.

I've seen practical uses for gamma GLMs discussed (with real data examples) in (off the top of my head) de Jong & Heller and Frees as well as numerous papers; I've also seen applications in other areas. Oh, and if I remember right, Venables and Ripley's MASS uses it on school absenteeism (the quine data; Edit: turns out it's actually in Statistics Complements to MASS, see p11, the 14th page of the pdf, it has a log link but there's a small shift of the DV). McCullagh and Nelder did a blood clotting example, though perhaps it may have been natural link.

Best resource for gamma GLM examples that I have seen so far is Dunn & Smyth (Generalized Linear Models With Examples in R, Springer), who discuss an example using forest biomass of small-leaved lime trees (and discuss both a linear model on the logs and a log-link gamma GLM, though fitting the GLM and comparing the results is left as an exercise); they also have a case study of yield density for onions (with inverse link); they have additional exercises with a response of insurance claim amount (log link), and another involving health care costs (log and identity links), as well as a number of other exercises where gamma GLMs are used and assessed.

Then there's Faraway's book where he did a car insurance example and a semiconductor manufacturing data example.

There are some advantages and some disadvantages to choosing either of the two options. Since these days both are easy to fit; it's generally a matter of choosing what's most suitable.

It's far from the only option; for example, there's also inverse Gaussian GLMs, which are more skew/heavier tailed (and even more heteroskedastic) than either gamma or lognormal.

As for drawbacks, it's harder to do prediction intervals. Some diagnostic displays are harder to interpret. Computing expectations on the scale of the linear predictor (generally the log-scale) is harder than for the equivalent lognormal model. Hypothesis tests and intervals are generally asymptotic. These are often relatively minor issues.

It has some advantages over log-link lognormal regression (taking logs and fitting an ordinary linear regression model); one is that mean prediction is easy. This is often a useful advantage for me.

Answer 2

That's a good question. In fact, why don't people use generalised linear models (GLM) more is also a good question.

Warning note: Some people use GLM for general linear model, not what is in mind here.

• It does depend where you look. For example, gamma distributions have been popular in several of the environmental sciences for some decades and so modelling with predictor variables too is a natural extension. There are many examples in hydrology and geomorphology, to name some fields in which I have strayed.

• It is hard to pin down quite when to use it beyond an empty answer of whenever it works best. Given skewed positive data I will often find myself trying gamma and lognormal models (in GLM context log link, normal or Gaussian family) and choosing which works better.

• Gamma modelling remained quite difficult to do until fairly recently, certainly as compared with say taking logs and applying linear regressions, without writing a lot of code yourself. Even now, I'd guess that it isn't equally easy across all the major statistical software environments.

• In explaining what is used and what is not used, despite merits and demerits, I think you always come down to precisely the kind of factors you identify: what is taught, what is in the literature that people read, what people hear talked about at work and at conferences. So, you need a kind of amateur sociology of science to explain. Most people seem to follow straight and narrow paths within their own fields. Loosely, the larger the internal literature in any field on modelling techniques, the less inclined people in that field seem to be to try something different.

Answer 3

Gamma regression is in the GLM and so you can get many useful quantities for diagnostic purposes, such as deviance residuals, leverages, Cook's distance, and so on. They are perhaps not as nice as the corresponding quantities for log-transformed data.

One thing that gamma regression avoids compared to the lognormal is transformation bias. Jensen's inequality implies that the predictions from lognormal regression will be systematically biased because it's modeling transformed data rather than the transformed expected value.

Also, gamma regression (or other models for nonnegative data) can cope with a broader array of data than the lognormal due to the fact that it can have a mode at 0, such as you have with the exponential distribution, which is in the gamma family, which is impossible for the lognormal.

I have read suggestions that using the Poisson likelihood as a quasi-likelihood is more stable. They're conjugates of each other. The quasi-Poisson also has the substantial benefit of being able to cope with exact 0 values, which trouble both the gamma and, especially, the lognormal.

Answer 4

In my opinion, it assumes that the errors lie on a family of gamma distributions, with the same shapes, and with the scales changing according the related formula.

But it is difficult to do model diagnosis. Note that the simple QQ plot is not suitable here, because it is about the same distribution, while ours is a family of distributions with different variances.

Naively, the residuals plot can be used to see that they have different scales but the same shape, usually with long tails.

In my experience, the gamma GLM may be tried for some long tail distributed problems, and it is widely used in insurance and environment sectors, etc. But the assumptions are difficult to test, and the model does not perform well usually, so different papers argue to use other family distributions with the same problem, like inverse Gaussian, etc. In practice, it seems that such choices depends on expert judgement with the industrial experience. This limits the use of the gamma GLM.