1
$\begingroup$

When you sum N IID samples from a univariate distribution, the sum approaches a normally-distributed value.

Likewise, when you sum variables together, the moment-generating functions of their distribution can be multiplied to produce the resulting moment-generating function of the sum.

It would seem that you could exponentiate a variable's moment-generating function by a large N, and the resulting function would be close to that of a normal distribution's.

However, from a mathematical perspective, this makes no sense, as you are simply

For simplicity's sake, let's use variables from continuous distributions that have an MGF (to avoid problems with the domain).

One example I am having trouble with is the Chi-squared distribution, with an MGF of

(1-2t)^(k/2)

The resulting MGF of the sum of many of these variables would be

(1-2t)^(N*k/2)

If we assume N*k/2 is an integer, C, then the MGF is

(1-2t)^C = 1-2t * C + 4t^2 * C*(C-1)/2 +...

Because C ≈ C-1 ≈ C-2 ≈ ... for large C, this can simplify to

sum((-2t*C)^n/n!) from n=0 to C ≈ e^(-2t*C)

Of course, the later terms deviate more strongly from the normal distribution's, but those derivatives don't affect the function's value very much.

Unfortunately, this MGF is degenerate, because it simply describes that of a variable that is always C. There is no t^2 term in the exponent, which is required for a normal MGF. Did I make a mistake in my calculation, or is there some fundamental assumption that I overlooked/violated?

Edit: I replaced "added" with "multiplied" in one instance because I was referring to multiplying the MGFs, not adding them.

Edit 2: I realized I forgot the minus sign in the exponent of the Chi-squared MGF. That resulted in a computational error.

$\endgroup$
  • 2
    $\begingroup$ You don't add $N$ MGFs together to find the MGF of the sum, you raise it to the $N^{th}$ power. Also: for large $C$, $C$ is not well-approximated by, say, 1 or 2, so your simplification is less than optimal. Try taking a Taylor expansion of the MGF around $t=0$, and see if you can fit it into the same general form (but with more terms) than the Normal MGF. Added note: $\log(1-2t) \approx -2t$ for $t$ small, so $(1-2t) \approx \exp\{-2t\}$. $\endgroup$ – jbowman Aug 7 '17 at 4:12
  • $\begingroup$ I made a typo. I meant to say multiply the MGFs together (which raises them to the Nth power). Also, what do you mean by C is not well-approximated by 1 or 2? $\endgroup$ – Max Candocia Aug 7 '17 at 12:45
  • $\begingroup$ $C \approx C-1 \approx C-2 \dots $ ends at $\approx 2 \approx 1$. But the answer below shows how it's done. $\endgroup$ – jbowman Aug 7 '17 at 13:52
  • $\begingroup$ Do those terms make that much of a difference, though? Those are very high orders of the Taylor expansion, where t is raised to a very large power. Also, after looking at the answer below, I realized I had a positive value in my exponent instead of a negative, which makes all the difference... $\endgroup$ – Max Candocia Aug 7 '17 at 14:49
3
$\begingroup$

When you sum $N$ IID samples from a univariate distribution, the sum approaches a normally-distributed value.

Not exactly. The sum doesn't converge. Under certain conditions, the average converges to the mean, and a scaled version of the de-meaned average converges to a normal distribution.

In particular, using your example, say you have an RV $X_i \sim \chi^2$, then its MGF is

$$ \left( 1 - 2t \right) ^{-\frac{k}{2}}. \; (1) $$

The second-order expansion of (1) is

$$ \sim \left( 1 - 2t \right) ^{-\frac{k}{2}}|_{t = 0} + \frac{\partial \left( 1 - 2t \right) ^{-\frac{k}{2}}}{\partial t}|_{t = 0}t + \frac{\partial^2 \left( 1 - 2t \right) ^{-\frac{k}{2}}}{2 \partial t^2}|_{t = 0}t^2 = 1 + k t + \frac{k}{2}(k + 2) t^2 $$

If you have $n$ independent RVs $X_1, \ldots, X_n$ with this distribution, then the MGF of $\sum_{i = 1}^n\left[x_i/n\right]$, which is just a linear combination, is therefore known to be

$$ \sim \left( 1 + k\frac{t}{n} + \frac{k}{2}(k + 2)\left(\frac{t}{n}\right)^2 \right)^n $$

The MGF of $\sqrt{n} \left(\sum_{i = 1}^n\left[x_i/n\right] - k\right)$, therefore, is

$$ \sim \left( 1 + \frac{\frac{k}{2}(k + 2)t^2}{n} \right)^n \sim e^{\frac{k}{2}(k + 2)t^2} $$

This is the MGF of a normal distribution.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.