# How many distributions are in the GLM?

I've identified multiple places in textbooks where the GLM is described with 5 distributions (viz., Gamma, Gaussian, Binomial, Inverse Gaussian, & Poisson). This is also exemplified in the family function in R.

Occasionally I come across references to the GLM where additional distributions are included (example). Can someone explain why these 5 are special or are always in the GLM but sometimes others are?

From what I've learned so far, the GLM distributions in the exponential family all fit into the form: $$f(y;\theta,\phi)=\exp\left\{\frac{y\theta-b(\theta)}{\phi}+c(y,\phi)\right\}$$ where $\phi$ is the dispersion parameter and $\theta$ is the canonical parameter.

Can't any distribution be transformed to fit in the GLM?

• Clearly, uniform distribution doesn't belong to exponential family. Jul 19 '15 at 1:49
• Nice question. E.g. what about the lognormal? Jul 21 '15 at 16:02
• @Zhanxiong, isn't uniform a special case of beta distribution, and beta distribution is in the exponential family? Jul 21 '15 at 22:03
• @shf8888 AFAIK it's only a exponential-family distribution in the limit, when it converges to the gamma distribution. Jul 21 '15 at 23:14
• Woah woah -- the right question isn't whether "uniform distribution is an exponential family distribution". It's whether the family of uniform distributions (e.g. all uniform distributions on intervals =(a,b) with a<b) is an exponential family. (It's not, because the support of the distribution changes as you change the parameters.) . But the family of beta distributions with 2 unknown parameters certainly is an exponential family, and the uniform distribution on (0,1) is a member of that family. Apr 5 '17 at 14:00

As you indicate, the qualification for using a distribution in a GLM is that it be of the exponential family (note: this is not the same thing as the exponential distribution! Although the exponential distribution, as a gamma distribution, is itself part of the exponential family). The five distributions you list are all of this family, and more importantly, are VERY common distributions, so they are used as examples and explanations.

As Zhanxiong notes, the uniform distribution (with unknown bounds) is a classic example of a non-exponential family distribution. shf8888 is confusing the general uniform distribution, on any interval, with a Uniform(0, 1). The Uniform(0,1) distribution is a special case of the beta distribution, which is an exponential family. Other non-exponential family distributions are mixture models and the t distribution.

You have the definition of the exponential family correct, and the canonical parameter is very important for using GLM. Still, I've always found it somewhat easier to understand the exponential family by writing it as:

$$f(x; \theta) = a(\theta)g(x)\exp\left[b(\theta)R(x)\right]$$

There is a more general way to write this, with a vector $\boldsymbol{\theta}$ instead of a scalar $\theta$; but the one-dimensional case explains a lot. Specifically, you must be able to factor your density's non-exponentiated part into two functions, one of unknown parameter $\theta$ but not observed data $x$ and one of $x$ and not $\theta$; and the same for the exponentiated part. It may be hard to see how, e.g., the binomial distribution can be written this way; but with some algebraic juggling, it becomes clear eventually.

We use the exponential family because it makes a lot of things much easier: for instance, finding sufficient statistics and testing hypotheses. In GLM, the canonical parameter is often used for finding a link function. Finally, a related illustration of why statisticians prefer to use the exponential family in just about every case is trying to do any classical statistical inference on, say, a Uniform($\theta_1$, $\theta_2$) distribution where both $\theta_1$ and $\theta_2$ are unknown. It's not impossible, but it's much more complicated and involved than doing the same for exponential family distributions.

• Beta distribution with both parameters unknown is still an exponential family (but a 2-parameter exponential family). What makes you think it isn't? www2.stat.duke.edu/courses/Spring11/sta114/lec/… or wikipedia Apr 5 '17 at 13:57
• Thanks for pointing this out, I've changed my comment...you're right! I really don't know what I meant Apr 6 '17 at 15:32