# Computing variance from moment generating function of exponential distribution

I'm wondering how to get variance of exp. distribution from the raw variance computed using the moment generating function. Here's my line of reasoning:

PDF of Exponential distriution is

$$p_X(x) = \lambda \cdot e^{-\lambda x}$$

for $$x > 0$$, and $$0$$ for $$x \leq 0$$.

Deriving the MGF:

\begin{aligned} M_X(t) &= \mathbb{E}\left[e^{t X}\right] && \text{definition} \\ &= \int_{- \infty}^{\infty} x \cdot p_X(x) dx&& \text{just definition of expectation} \\ &= \int_{- \infty}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{LOTUS} \\ &= \int_{0}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{since } x > 0 \\ &= \lambda \int_{0}^{\infty} e^{t x} \cdot e^{-\lambda x} dx&& \text{the constant multiple rule} \\ &= \lambda \int_{0}^{\infty} e^{t x -\lambda x} dx \\ &= \lambda \int_{0}^{\infty} e^{x (t -\lambda )} dx \\ &= \lambda \cdot \frac{1}{\lambda - t} && \text{closed form solution for } t < \lambda \\ &= \frac{\lambda}{\lambda - t} \qquad \boxed{\checkmark} \text{ Wikipedia check} \end{aligned}

Getting moments of exponential distributions by derivating MGF

$$M_X(t) = \frac{\lambda}{\lambda - t}$$

First moment (expectation)

$$M_X^{(1)}(t) = \frac{\partial}{\partial t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{\lambda}{(\lambda - t)^2}$$

• And evaluate at $$t = 0$$:

$$\frac{\lambda}{(\lambda - t)^2} \bigg\vert_{t=0} = \frac{\lambda}{\lambda^2} = \frac{1}{\lambda} \qquad \boxed{\checkmark} \text{ Wikipedia check}$$

Second moment

$$M_X^{(2)}(t) = \frac{\partial^2}{\partial^2 t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{2 \lambda}{(\lambda - t)^3}$$

$$\frac{2 \lambda}{(\lambda - t)^3} \bigg\vert_{t=0} = \frac{2}{\lambda^2}$$

So this is raw variance but not the actual variance $$\frac{1}{\lambda^2}$$... how to get there?

$$M_X^{(2)}(0)$$ is not a variance, it is $$E(X^2)$$. So the variance can be obtained by $$Var(X) = E(X^2) - E(X)^2 = M_X^{(2)}(0) - [M_X^{(1)}(0)]^2 = \frac{1}{\lambda^2}$$

• Thanks! So I guess my confusion was due to terminological complexity of moments (raw, central, normalised), as shown for example in the table here en.wikipedia.org/wiki/Moment_(mathematics) Oct 29 '20 at 8:17
• There is another useful function related to mgf, which is called a cumulant generating function(cgf, $C_X(t)$). cgf is defined as $C_X(t) = \log M_X(t)$ and its first derivative and second derivative evaluated at $t=0$ are mean and variance respectively. In multivariate cases, we can easily get covariance from cgf. en.wikipedia.org/wiki/Cumulant Oct 29 '20 at 9:00
• You just showed me another piece of puzzle that I was missing somewhere else :) Thanks! Oct 29 '20 at 15:15

The second moment gives you

$$E[X^2]$$

and the variance is defined as

$$E[X^2]-E[X]^2$$

so that you get

$$2/\lambda^2-(1/\lambda)^2$$

which will then give you the desired result.

• Yes, thanks, it's now clear to me that the second moment does not equal variance (as apposed to the first moment which is the expectation). Oct 29 '20 at 8:44
• Nitpick: the variance is defined as $$E[(X−E[X])^2]$$ and from this one derives the more used formula in the answer. Oct 29 '20 at 18:11