9
$\begingroup$

I'm wondering how to get variance of exp. distribution from the raw variance computed using the moment generating function. Here's my line of reasoning:

PDF of Exponential distriution is

$$ p_X(x) = \lambda \cdot e^{-\lambda x} $$

for $x > 0$, and $0$ for $x \leq 0$.

Deriving the MGF:

$$ \begin{aligned} M_X(t) &= \mathbb{E}\left[e^{t X}\right] && \text{definition} \\ &= \int_{- \infty}^{\infty} x \cdot p_X(x) dx&& \text{just definition of expectation} \\ &= \int_{- \infty}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{LOTUS} \\ &= \int_{0}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{since } x > 0 \\ &= \lambda \int_{0}^{\infty} e^{t x} \cdot e^{-\lambda x} dx&& \text{the constant multiple rule} \\ &= \lambda \int_{0}^{\infty} e^{t x -\lambda x} dx \\ &= \lambda \int_{0}^{\infty} e^{x (t -\lambda )} dx \\ &= \lambda \cdot \frac{1}{\lambda - t} && \text{closed form solution for } t < \lambda \\ &= \frac{\lambda}{\lambda - t} \qquad \boxed{\checkmark} \text{ Wikipedia check} \end{aligned} $$

Getting moments of exponential distributions by derivating MGF

$$ M_X(t) = \frac{\lambda}{\lambda - t} $$

First moment (expectation)

$$ M_X^{(1)}(t) = \frac{\partial}{\partial t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{\lambda}{(\lambda - t)^2} $$

  • And evaluate at $t = 0$:

$$ \frac{\lambda}{(\lambda - t)^2} \bigg\vert_{t=0} = \frac{\lambda}{\lambda^2} = \frac{1}{\lambda} \qquad \boxed{\checkmark} \text{ Wikipedia check} $$

Second moment

$$ M_X^{(2)}(t) = \frac{\partial^2}{\partial^2 t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{2 \lambda}{(\lambda - t)^3} $$

$$ \frac{2 \lambda}{(\lambda - t)^3} \bigg\vert_{t=0} = \frac{2}{\lambda^2} $$

So this is raw variance but not the actual variance $\frac{1}{\lambda^2}$... how to get there?

$\endgroup$
15
$\begingroup$

$M_X^{(2)}(0)$ is not a variance, it is $E(X^2)$. So the variance can be obtained by $$Var(X) = E(X^2) - E(X)^2 = M_X^{(2)}(0) - [M_X^{(1)}(0)]^2 = \frac{1}{\lambda^2}$$

$\endgroup$
3
  • $\begingroup$ Thanks! So I guess my confusion was due to terminological complexity of moments (raw, central, normalised), as shown for example in the table here en.wikipedia.org/wiki/Moment_(mathematics) $\endgroup$
    – John Doe
    Oct 29 '20 at 8:17
  • 2
    $\begingroup$ There is another useful function related to mgf, which is called a cumulant generating function(cgf, $C_X(t)$). cgf is defined as $C_X(t) = \log M_X(t)$ and its first derivative and second derivative evaluated at $t=0$ are mean and variance respectively. In multivariate cases, we can easily get covariance from cgf. en.wikipedia.org/wiki/Cumulant $\endgroup$
    – flossy
    Oct 29 '20 at 9:00
  • $\begingroup$ You just showed me another piece of puzzle that I was missing somewhere else :) Thanks! $\endgroup$
    – John Doe
    Oct 29 '20 at 15:15
6
$\begingroup$

The second moment gives you

$$E[X^2]$$

and the variance is defined as

$$E[X^2]-E[X]^2$$

so that you get

$$2/\lambda^2-(1/\lambda)^2$$

which will then give you the desired result.

$\endgroup$
2
  • $\begingroup$ Yes, thanks, it's now clear to me that the second moment does not equal variance (as apposed to the first moment which is the expectation). $\endgroup$
    – John Doe
    Oct 29 '20 at 8:44
  • $\begingroup$ Nitpick: the variance is defined as $$E[(X−E[X])^2]$$ and from this one derives the more used formula in the answer. $\endgroup$ Oct 29 '20 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.