# Relationship between exponential families and moment generating functions

I have recently been playing around with some change of measure arguments for shifting the mean of a sub-gaussian distribution. It occurred to me however, that sub-gaussianity might not be the natural property to consider, but rather being a member of an exponential family (as this simplifies the likelihood transform induced by an exponential tilting).

Now, I must admit that I do not know much about these, but I encountered a rather miraculous claim on wikipedia, https://en.wikipedia.org/wiki/Natural_exponential_family:

"Every distribution possessing a moment-generating function is a member of a natural exponential family."

As I figured it fair to try to read up a bit before asking stupid questions, I familiarized myself with a paper which seems to prove this result partially:

SAMPSON, Allan R. Characterizing exponential family distributions by moment generating functions. The Annals of Statistics, 1975, 747-753.

Here, the author shows that if the moment generating function is of a particular form, then, the distribution is a member of an exponential family. There however still seems to be a leap, one which I am not able to make by myself, to the claim I quoted; in particular, the hypotheses of the claim seem to be weaker and the conclusion stronger (as we have narrowed down to a natural exponential family)!

I would very much appreciate a reference on this result, preferably with proof (as it seems to me an amazing property which I cannot entirely wrap my head around...) Of course, if anyone knows the proof, I would be even happier if that person answered. Thank you in advance!

• It is obviously a wrong statement. Every distribution with a compact support enjoys a moment generating function, since $\exp\{tX\}$ is bounded and hence $\mathbb{E}[\exp\{tX\}]$ is finite for all $t$'s. Jan 18 '19 at 12:28
• @Xi'an, thank you for your reply. However, I don't see how your claim disproves the claim in the wikipedia article. It is my understanding that distributions with compact support may well be part of an exponential family - it is just that they do not remain the same family under transformations of the natural parameters. I am NOT asking about it being a closed family in some sense (which yes, clearly is wrong)! I am entirely content with one parameter of a certain family of distributions (say uniform) being in one exponential family for one parameter value and another for the second. Jan 18 '19 at 15:03
• @Xi'an OK, now I see your point. As I said I am not well-versed in these matters (I was trying to make you make me see the contradiction/sillyness)... I guess the wikipedia claim really is \emph{strange}. If you feel like summarizing your comments in an answer, I will accept them. Thanks for your time again! Jan 19 '19 at 9:05

To link the existence of the moment-generating function and the inclusion in an exponential family is absurd in that the former is a property of a given single distribution while the later is a characteristic of a family of distributions parameterised with a finite dimensional parameter. Note that a given arbitrary distribution with density $$f_0$$ can always induce an exponential family by considering an arbitrary transform $$T$$ and the family of densities $$f_\theta(x|)=f_0(x)\exp\{T(x)\cdot\theta-\psi(\theta)\}$$ In particular, the exponentially tilded measure is defined by $$f_θ(x) = \dfrac{\exp\{θ\cdot x\}f_0(x)}{M_X(θ)}$$ where $$M_X(\theta)$$ is the moment-generating function for $$X$$.
Furthermore, the moment-generating function is associated with a specific representation of a distribution, namely, $$\mathbb{E}^X[\exp\{t X\}]$$ differs from $$\mathbb{E}^X[\exp\{t \xi(X)\}]$$, while exponential families are equivariant under one-to-one transforms. Moment-generating functions can be found for all distributions with compacts support (or alternatively for transforms $$\xi(\cdot)$$ taking values in a compact set). They also exist for some families of distributions that do not constitute an exponential family, like the family of double-exponential or Laplace distributions, and the family of non-central chi-squared distributions.