9
$\begingroup$

I'm wondering how to get variance of exp. distribution from the raw variance computed using the moment generating function. Here's my line of reasoning:

PDF of Exponential distriution is

$$ p_X(x) = \lambda \cdot e^{-\lambda x} $$

for $x > 0$, and $0$ for $x \leq 0$.

Deriving the MGF:

$$ \begin{aligned} M_X(t) &= \mathbb{E}\left[e^{t X}\right] && \text{definition} \\ &= \int_{- \infty}^{\infty} x \cdot p_X(x) dx&& \text{just definition of expectation} \\ &= \int_{- \infty}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{LOTUS} \\ &= \int_{0}^{\infty} e^{t x} \cdot \lambda e^{-\lambda x} dx&& \text{since } x > 0 \\ &= \lambda \int_{0}^{\infty} e^{t x} \cdot e^{-\lambda x} dx&& \text{the constant multiple rule} \\ &= \lambda \int_{0}^{\infty} e^{t x -\lambda x} dx \\ &= \lambda \int_{0}^{\infty} e^{x (t -\lambda )} dx \\ &= \lambda \cdot \frac{1}{\lambda - t} && \text{closed form solution for } t < \lambda \\ &= \frac{\lambda}{\lambda - t} \qquad \boxed{\checkmark} \text{ Wikipedia check} \end{aligned} $$

Getting moments of exponential distributions by derivating MGF

$$ M_X(t) = \frac{\lambda}{\lambda - t} $$

First moment (expectation)

$$ M_X^{(1)}(t) = \frac{\partial}{\partial t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{\lambda}{(\lambda - t)^2} $$

  • And evaluate at $t = 0$:

$$ \frac{\lambda}{(\lambda - t)^2} \bigg\vert_{t=0} = \frac{\lambda}{\lambda^2} = \frac{1}{\lambda} \qquad \boxed{\checkmark} \text{ Wikipedia check} $$

Second moment

$$ M_X^{(2)}(t) = \frac{\partial^2}{\partial^2 t} \left( \frac{\lambda}{\lambda - t} \right) = \frac{2 \lambda}{(\lambda - t)^3} $$

$$ \frac{2 \lambda}{(\lambda - t)^3} \bigg\vert_{t=0} = \frac{2}{\lambda^2} $$

So this is raw variance but not the actual variance $\frac{1}{\lambda^2}$... how to get there?

$\endgroup$

2 Answers 2

16
$\begingroup$

$M_X^{(2)}(0)$ is not a variance, it is $E(X^2)$. So the variance can be obtained by $$Var(X) = E(X^2) - E(X)^2 = M_X^{(2)}(0) - [M_X^{(1)}(0)]^2 = \frac{1}{\lambda^2}$$

$\endgroup$
3
  • $\begingroup$ Thanks! So I guess my confusion was due to terminological complexity of moments (raw, central, normalised), as shown for example in the table here en.wikipedia.org/wiki/Moment_(mathematics) $\endgroup$
    – John Doe
    Commented Oct 29, 2020 at 8:17
  • 2
    $\begingroup$ There is another useful function related to mgf, which is called a cumulant generating function(cgf, $C_X(t)$). cgf is defined as $C_X(t) = \log M_X(t)$ and its first derivative and second derivative evaluated at $t=0$ are mean and variance respectively. In multivariate cases, we can easily get covariance from cgf. en.wikipedia.org/wiki/Cumulant $\endgroup$
    – flossy
    Commented Oct 29, 2020 at 9:00
  • $\begingroup$ You just showed me another piece of puzzle that I was missing somewhere else :) Thanks! $\endgroup$
    – John Doe
    Commented Oct 29, 2020 at 15:15
6
$\begingroup$

The second moment gives you

$$E[X^2]$$

and the variance is defined as

$$E[X^2]-E[X]^2$$

so that you get

$$2/\lambda^2-(1/\lambda)^2$$

which will then give you the desired result.

$\endgroup$
2
  • $\begingroup$ Yes, thanks, it's now clear to me that the second moment does not equal variance (as apposed to the first moment which is the expectation). $\endgroup$
    – John Doe
    Commented Oct 29, 2020 at 8:44
  • $\begingroup$ Nitpick: the variance is defined as $$E[(X−E[X])^2]$$ and from this one derives the more used formula in the answer. $\endgroup$ Commented Oct 29, 2020 at 18:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.