Proof that moment generating functions uniquely determine probability distributions

Question

Wackerly et al's text states this theorem "Let $m_x(t)$ and $m_y(t)$ denote the moment-generating functions of random variables X and Y, respectively. If both moment-generating functions exist and $m_x(t) = m_y(t)$ for all values of t, then X and Y have the same probability distribution." without a proof saying its beyond the scope of the text. Scheaffer Young also has the same theorem without a proof. I don't have a copy of Casella, but Google book search didn't seem to find the theorem in it.

Gut's text seems to have an outline of a proof, but doesn't make reference to the "well-known results" and also requires knowing another result whose proof is also not provided.

Does anyone know who originally proved this and if the proof is available online anywhere? Otherwise how would one fill in the details of this proof?

In case I get asked no this is not a homework question, but I could imagine this possibly being someone's homework. I took a course sequence based on the Wackerly text and I have been left wondering about this proof for some time. So I figured it was just time to ask.

Answer 1

The general proof of this can be found in Feller (An Introduction to Probability Theory and Its Applications, Vol. 2). It is an inversion problem involving Laplace transform theory. Did you notice that the mgf bears a striking resemblance to the Laplace transform?. For use of Laplace Transformation you can see Widder (Calcus Vol I) .

Proof of a special case:

Suppose that X and Y are random varaibles both taking only possible values in {$0, 1, 2,\dots, n$}. Further, suppose that X and Y have the same mgf for all t: $$\sum_{x=0}^ne^{tx}f_X(x)=\sum_{y=0}^ne^{ty}f_Y(y)$$ For simplicity, we will let $s = e^t$ and we will define $c_i = f_X(i) − f_Y (i)$ for $i = 0, 1,\dots,n$.

Now $$\sum_{x=0}^ne^{tx}f_X(x)-\sum_{y=0}^ne^{ty}f_Y(y)=0$$ $$\Rightarrow \sum_{x=0}^ns^xf_X(x)-\sum_{y=0}^ns^yf_Y(y)=0$$ $$\Rightarrow \sum_{x=0}^ns^xf_X(x)-\sum_{x=0}^ns^xf_Y(x)=0$$ $$\Rightarrow\sum_{x=0}^ns^x[f_X(x)-f_Y(x)]=0$$ $$\Rightarrow \sum_{x=0}^ns^xc_x=0~∀s>0$$ The above is simply a polynomial in s with coefficients $c_0, c_1,\dots,c_n$. The only way it can be zero for all values of s is if $c_0=c_1=\cdots= c_n=0$.So, we have that $0=c_i=f_X(i)−f_Y(i)$ for $i=0, 1,\dots,n$.

Therefore, $f_X(i)=f_Y(i)$ for $i=0,1,\dots,n$.

In other words the density functions for $X$ and $Y$ are exactly the same. In other other words, $X$ and $Y$ have the same distributions.

Answer 2

The theorem you are discussing is a basic result in probability/measure theory. The proofs would more likely be found in books on probability or statistical theory. I found the analogous result for characteristic functions given in Hoel Port and Stone pp 205-208

Tucker pp 51-53

and Chung pp 151-155 This is the Third Edition. I have the second edition and am referring to the page numbers in the second edition published in 1974.

The proof for the mgf I found to be more difficult to find but you can find it in Billingley's book "Probability and Measure" pp. 342-345. On page 342 Theorem 30.1 provides the theorem that answers the moment problem. On page 345 Billingsley states the result that if a probability measure has a moment generating function M(s) defined on an interval surrounding 0 then the hypothesis for Theorem 30.1 is satisfied and hence the measure is determined by its moments. But these moment s are determined by M(s). Hence the measure is determined by its moment generating function if M(s) exists in a neighborhood of 0. So this logic along with the proof he gives for Theorem30.1 proves the result. Billingsley also comment that the solution to exercise 26.7 on page 305 is an alternative proof of the uniqueness theorem for moment generating functions.

Answer 3

Denote the moment generating function of $X$ by $M_X(t)=Ee^{tX}$.

Uniqueness Theorem. If there exists $\delta>0$ such that $M_X(t) = M_Y(t) < \infty$ for all $t \in (-\delta,\delta)$, then $F_X(t) = F_Y(t)$ for all $t \in \mathbb{R}$.

To prove that the moment generating function determines the distribution, there are at least two approaches:

• To show that finiteness of $M_X$ on $(-\delta,\delta)$ implies that the moments $X$ do not increase too fast, so that $F_X$ is determined by $(EX^k)_{k\in\mathbb{N}}$, which are in turn determined by $M_X$. This proof can be found in Section 30 of Billingsley, P. Probability and Measure.

• To show that $M_X$ is analytic and can be extended to $(-\delta,\delta)\times i\mathbb{R} \subseteq \mathbb{C}$, so that $M_X(z)=Ee^{zX}$, so in particular $M_X(it)=\varphi_X(t)$ for all $t\in\mathbb{R}$, and then use the fact that $\varphi_X$ determines $F_X$. For this approach, see Curtiss, J. H. Ann. Math. Statistics 13:430-433 and references therein.

At undergraduate level, almost every textbook works with the moment generating function and states the above theorem without proving it. It makes sense, because the proof requires far more advanced mathematics than undergraduate level allows.

At the point when students have all the tools needed in the proof, they also have the maturity to work with the characteristic function $\varphi_X(t)=Ee^{itX}$ instead. Almost every graduate textbook takes this path, they prove that the characteristic function determines the distribution and basically ignore moment generating functions altogether.