I have come across the term exponential family.

The Bernoulli, Gaussian and many more distributions come under this exponential family.

What would be the commonalities between them?


They're called "exponential family" because they can all be written in the simple form

$$f_X(x|\theta) = \exp \left (\eta(\theta) \cdot T(x) - A(\theta) + B(x) \right )$$

There are some other, equivalent forms that are perhaps more often used, but I think that form makes the "exponential" part clearest.

See, for example, this section of the Wikipedia page on the Exponential family.

In particular, in the mid 1930s a number of authors discussed what conditions would be required for a distribution to have a sufficient statistic; Koopman$^{[1]}$ stated it would have to be "of the very special exponential type of formula (4) below" (emphasis mine), where equation (4) was equivalent to the above form.

So that form succinctly expresses what they all have in common. But the consequence of that form is that this particular class of distributions has some very nice properties; for example, $T$ is a sufficient statistic - it carries all of the information in the data about $\theta$.

A number of additional properties they all share are summarized here.

Commonly used members include the Gaussian, Poisson, binomial, and gamma (including exponential and chi-squared), but I've also had occasion to use other members (such as Tweedie with specified $p$, and inverse Gaussian).

The shared properties make some standardization of the treatment of them possible, leading to the wide use of generalized linear models (GLMs).

[1] Koopman, B.O., (1936),
"On Distributions Admitting a Sufficient Statistic",
Transactions of the American Mathematical Society, 39:3, 399-409.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.