3
$\begingroup$

I'm trying to understand how the Gamma distribution, which is always positive, is used to describe errors when using a GLM. In practice, errors can be negative, as I get negative residuals when applying a GLM with a Gamma distribution. Is it that the Gamma distribution describes the general shape of the error distribution, but the mean can be shifted so as to give negative errors? Clearly I've misunderstood something fundamental, but I'm not sure what that is...

$\endgroup$
4
$\begingroup$

The Gamma distribution doesn't describe the errors, it describes the conditional distribution of $Y$.

$(Y|X=\underline{x})\sim\text{Gamma}$

Indeed, it's only occasionally useful to think of GLMs as having 'errors' at all. The data have distributions from some family, but the distributions vary because of a model that describes the mean as a transformed linear function of predictors.

The wikipedia page describes the components of a GLM here. You will find mention of 'errors' in terms of the model a couple of times on that page, but they're not discussed in any very substantive way (for good reason).

An alternative would be something along the lines of John Fox's book on Applied Regression Analysis and Generalized Linear Models. The publisher has made three chapters of that book available, including the intro chapter on GLMs. I'd advise reading at least the first 7 or so of that.

If you subtract the estimate of the mean from a Gamma random variable, of course you end up with some negative values. (If the assumptions all apply, and the mean was known rather than estimated, you you'd end up with residuals that were a collection of shifted Gammas)

In the Gaussian case, there's a close connection between the conditional distribution and errors, but that kind of thing really only makes much sense with a location-family. Most distributions in the exponential class are not location-families.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the reply, Glen. I must admit, I'm still confused. My understanding is largely derived from the R documentation, which states that the family parameter in the glm function (where the Gamma distribution is chosen) is "a description of the error distribution and link function to be used in the model". Is this just a poor explanation in the documentation, then? $\endgroup$ – Lyngbakr Mar 5 '15 at 23:30
  • $\begingroup$ Well, it's certainly not the explanation I'd have used. I'll include a link n my answer. $\endgroup$ – Glen_b Mar 6 '15 at 1:35
  • $\begingroup$ Some links are there now. I'd rather have said something like the family is the conditional distribution of the response (y) variable. $\endgroup$ – Glen_b Mar 6 '15 at 1:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.