26
$\begingroup$

Wackerly et al's text states this theorem "Let $m_x(t)$ and $m_y(t)$ denote the moment-generating functions of random variables X and Y, respectively. If both moment-generating functions exist and $m_x(t) = m_y(t)$ for all values of t, then X and Y have the same probability distribution." without a proof saying its beyond the scope of the text. Scheaffer Young also has the same theorem without a proof. I don't have a copy of Casella, but Google book search didn't seem to find the theorem in it.

Gut's text seems to have an outline of a proof, but doesn't make reference to the "well-known results" and also requires knowing another result whose proof is also not provided.

Does anyone know who originally proved this and if the proof is available online anywhere? Otherwise how would one fill in the details of this proof?

In case I get asked no this is not a homework question, but I could imagine this possibly being someone's homework. I took a course sequence based on the Wackerly text and I have been left wondering about this proof for some time. So I figured it was just time to ask.

$\endgroup$
5
  • 3
    $\begingroup$ Related: (i) Inversion of mgfs and (ii) Existence of the moment generating function and variance $\endgroup$ – cardinal Aug 23 '12 at 16:30
  • 3
    $\begingroup$ If you have access to Billingsley's Probability and Measure text, this is discussed in a section entitled, I believe, "The method of moments". (Apologies for the vagueness, as I don't currently have it at hand.) If I recall correctly, the proof he uses relies on the corresponding results for characteristic functions, though, which may not be completely satisfying. This is certainly (well) outside the scope of the expected background of Wackerly's text. $\endgroup$ – cardinal Aug 23 '12 at 16:38
  • 1
    $\begingroup$ Wow @cardinal your answers to those questions were superior and very helpful thank you and thanks for the text recommendation I should get a hold of a copy. $\endgroup$ – Chris Simokat Aug 23 '12 at 18:00
  • 2
    $\begingroup$ Concerning the history ("who originally proved this?"), it appears Laplace was using the characteristic function for this kind of work in 1785 and had developed the general inversion formula (which is the key to the proof) by 1810. See Anders Hald, A History of mathematical Statistics from 1750 to 1930, chapter 17. $\endgroup$ – whuber Aug 23 '12 at 22:49
  • $\begingroup$ Just a remark that the 'other result' is proved in Gut, since this property is a consequence of the inversion formula which is proved there. $\endgroup$ – user334639 Aug 3 '17 at 14:51
28
$\begingroup$

The general proof of this can be found in Feller (An Introduction to Probability Theory and Its Applications, Vol. 2). It is an inversion problem involving Laplace transform theory. Did you notice that the mgf bears a striking resemblance to the Laplace transform?. For use of Laplace Transformation you can see Widder (Calcus Vol I) .

Proof of a special case:

Suppose that X and Y are random varaibles both taking only possible values in {$0, 1, 2,\dots, n$}. Further, suppose that X and Y have the same mgf for all t: $$\sum_{x=0}^ne^{tx}f_X(x)=\sum_{y=0}^ne^{ty}f_Y(y)$$ For simplicity, we will let $s = e^t$ and we will define $c_i = f_X(i) − f_Y (i)$ for $i = 0, 1,\dots,n$.

Now $$\sum_{x=0}^ne^{tx}f_X(x)-\sum_{y=0}^ne^{ty}f_Y(y)=0$$ $$\Rightarrow \sum_{x=0}^ns^xf_X(x)-\sum_{y=0}^ns^yf_Y(y)=0$$ $$\Rightarrow \sum_{x=0}^ns^xf_X(x)-\sum_{x=0}^ns^xf_Y(x)=0$$ $$\Rightarrow\sum_{x=0}^ns^x[f_X(x)-f_Y(x)]=0$$ $$\Rightarrow \sum_{x=0}^ns^xc_x=0~∀s>0$$ The above is simply a polynomial in s with coefficients $c_0, c_1,\dots,c_n$. The only way it can be zero for all values of s is if $c_0=c_1=\cdots= c_n=0$.So, we have that $0=c_i=f_X(i)−f_Y(i)$ for $i=0, 1,\dots,n$.

Therefore, $f_X(i)=f_Y(i)$ for $i=0,1,\dots,n$.

In other words the density functions for $X$ and $Y$ are exactly the same. In other other words, $X$ and $Y$ have the same distributions.

$\endgroup$
1
  • 1
    $\begingroup$ Mainly The Moment Generating Function Uniquely Determines the Distribution. $\endgroup$ – Argha Aug 23 '12 at 18:09
9
$\begingroup$

The theorem you are discussing is a basic result in probability/measure theory. The proofs would more likely be found in books on probability or statistical theory. I found the analogous result for characteristic functions given in Hoel Port and Stone pp 205-208

Tucker pp 51-53

and Chung pp 151-155 This is the Third Edition. I have the second edition and am referring to the page numbers in the second edition published in 1974.

The proof for the mgf I found to be more difficult to find but you can find it in Billingley's book "Probability and Measure" pp. 342-345. On page 342 Theorem 30.1 provides the theorem that answers the moment problem. On page 345 Billingsley states the result that if a probability measure has a moment generating function M(s) defined on an interval surrounding 0 then the hypothesis for Theorem 30.1 is satisfied and hence the measure is determined by its moments. But these moment s are determined by M(s). Hence the measure is determined by its moment generating function if M(s) exists in a neighborhood of 0. So this logic along with the proof he gives for Theorem30.1 proves the result. Billingsley also comment that the solution to exercise 26.7 on page 305 is an alternative proof of the uniqueness theorem for moment generating functions.

$\endgroup$
3
  • 6
    $\begingroup$ Where is this in Chung? Did you mean pages 161-165, by chance? Even so, that deals with characteristic functions, not moment-generating functions, as requested by the OP. $\endgroup$ – cardinal Aug 23 '12 at 16:36
  • 1
    $\begingroup$ @cardinal Yes I know. I mentioned the result for characteristic functions because that is what i have found thus far. As I said the page numbers in Chung are based on the second edition that i have. I do not know where it appears in the third edition. I think there should be some sources that will have the result for mgfs. $\endgroup$ – Michael R. Chernick Aug 23 '12 at 17:07
  • 1
    $\begingroup$ I upvoted, because I appreciate your answer too so thank you for taking the time. $\endgroup$ – Chris Simokat Aug 24 '12 at 15:06
5
$\begingroup$

Denote the moment generating function of $X$ by $M_X(t)=Ee^{tX}$.

Uniqueness Theorem. If there exists $\delta>0$ such that $M_X(t) = M_Y(t) < \infty$ for all $t \in (-\delta,\delta)$, then $F_X(t) = F_Y(t)$ for all $t \in \mathbb{R}$.

To prove that the moment generating function determines the distribution, there are at least two approaches:

  • To show that finiteness of $M_X$ on $(-\delta,\delta)$ implies that the moments $X$ do not increase too fast, so that $F_X$ is determined by $(EX^k)_{k\in\mathbb{N}}$, which are in turn determined by $M_X$. This proof can be found in Section 30 of Billingsley, P. Probability and Measure.

  • To show that $M_X$ is analytic and can be extended to $(-\delta,\delta)\times i\mathbb{R} \subseteq \mathbb{C}$, so that $M_X(z)=Ee^{zX}$, so in particular $M_X(it)=\varphi_X(t)$ for all $t\in\mathbb{R}$, and then use the fact that $\varphi_X$ determines $F_X$. For this approach, see Curtiss, J. H. Ann. Math. Statistics 13:430-433 and references therein.

At undergraduate level, almost every textbook works with the moment generating function and states the above theorem without proving it. It makes sense, because the proof requires far more advanced mathematics than undergraduate level allows.

At the point when students have all the tools needed in the proof, they also have the maturity to work with the characteristic function $\varphi_X(t)=Ee^{itX}$ instead. Almost every graduate textbook takes this path, they prove that the characteristic function determines the distribution and basically ignore moment generating functions altogether.

$\endgroup$
4
  • $\begingroup$ Today, mgfs shouldnt be ignored as thry are much more useful numerically than the characteristic function $\endgroup$ – kjetil b halvorsen Aug 10 '17 at 13:11
  • 1
    $\begingroup$ Indeed! And yet I have never seen a textbook that emphasizes numerical methods but has deep enough math to give a proof of the Uniqueness Theorem. $\endgroup$ – user334639 Aug 11 '17 at 18:01
  • $\begingroup$ Curtiss' approach is exercise 26.7 in Billingsley (3rd edition) $\endgroup$ – Peter Arndt Dec 20 '20 at 13:12
  • $\begingroup$ @PeterArndt Yes, indeed. But it is quite convoluted, and it refers to 21.17, which in turn refers to 16.6. It would be good to see a complete sketch of how to "solve" this "exercise". $\endgroup$ – user334639 Dec 29 '20 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.