2
$\begingroup$

I have recently been playing around with some change of measure arguments for shifting the mean of a sub-gaussian distribution. It occurred to me however, that sub-gaussianity might not be the natural property to consider, but rather being a member of an exponential family (as this simplifies the likelihood transform induced by an exponential tilting).

Now, I must admit that I do not know much about these, but I encountered a rather miraculous claim on wikipedia, https://en.wikipedia.org/wiki/Natural_exponential_family:

"Every distribution possessing a moment-generating function is a member of a natural exponential family."

As I figured it fair to try to read up a bit before asking stupid questions, I familiarized myself with a paper which seems to prove this result partially:

SAMPSON, Allan R. Characterizing exponential family distributions by moment generating functions. The Annals of Statistics, 1975, 747-753.

Here, the author shows that if the moment generating function is of a particular form, then, the distribution is a member of an exponential family. There however still seems to be a leap, one which I am not able to make by myself, to the claim I quoted; in particular, the hypotheses of the claim seem to be weaker and the conclusion stronger (as we have narrowed down to a natural exponential family)!

I would very much appreciate a reference on this result, preferably with proof (as it seems to me an amazing property which I cannot entirely wrap my head around...) Of course, if anyone knows the proof, I would be even happier if that person answered. Thank you in advance!

$\endgroup$
3
  • $\begingroup$ It is obviously a wrong statement. Every distribution with a compact support enjoys a moment generating function, since $\exp\{tX\}$ is bounded and hence $\mathbb{E}[\exp\{tX\}]$ is finite for all $t$'s. $\endgroup$
    – Xi'an
    Jan 18, 2019 at 12:28
  • 1
    $\begingroup$ @Xi'an, thank you for your reply. However, I don't see how your claim disproves the claim in the wikipedia article. It is my understanding that distributions with compact support may well be part of an exponential family - it is just that they do not remain the same family under transformations of the natural parameters. I am NOT asking about it being a closed family in some sense (which yes, clearly is wrong)! I am entirely content with one parameter of a certain family of distributions (say uniform) being in one exponential family for one parameter value and another for the second. $\endgroup$ Jan 18, 2019 at 15:03
  • 1
    $\begingroup$ @Xi'an OK, now I see your point. As I said I am not well-versed in these matters (I was trying to make you make me see the contradiction/sillyness)... I guess the wikipedia claim really is \emph{strange}. If you feel like summarizing your comments in an answer, I will accept them. Thanks for your time again! $\endgroup$ Jan 19, 2019 at 9:05

1 Answer 1

2
$\begingroup$

To link the existence of the moment-generating function and the inclusion in an exponential family is absurd in that the former is a property of a given single distribution while the later is a characteristic of a family of distributions parameterised with a finite dimensional parameter. Note that a given arbitrary distribution with density $f_0$ can always induce an exponential family by considering an arbitrary transform $T$ and the family of densities $$f_\theta(x|)=f_0(x)\exp\{T(x)\cdot\theta-\psi(\theta)\}$$ In particular, the exponentially tilded measure is defined by $$f_θ(x) = \dfrac{\exp\{θ\cdot x\}f_0(x)}{M_X(θ)}$$ where $M_X(\theta)$ is the moment-generating function for $X$.

Furthermore, the moment-generating function is associated with a specific representation of a distribution, namely, $\mathbb{E}^X[\exp\{t X\}]$ differs from $\mathbb{E}^X[\exp\{t \xi(X)\}]$, while exponential families are equivariant under one-to-one transforms. Moment-generating functions can be found for all distributions with compacts support (or alternatively for transforms $\xi(\cdot)$ taking values in a compact set). They also exist for some families of distributions that do not constitute an exponential family, like the family of double-exponential or Laplace distributions, and the family of non-central chi-squared distributions.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you for an excellent clarification. Accepted your answer and will upvote as soon as I exceed 15... $\endgroup$ Jan 19, 2019 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.