2
$\begingroup$

The central limit theorem states that the limiting distribution of a centered and normalized sum of independent random variables with mean $\mu$ and finite variance $\sigma^2$ is Gaussian.

$$ \frac{\sum_{i=1}^n(X_i-\mu)}{\sigma\sqrt{n}}\xrightarrow{d}N(0,1) $$

However in practice, we may not be working with sums of centered and normalized random variables. Still, if we run experiments where we sum without normalization, the distribution of the sum can look increasingly Gaussian with increasing mean and variance. The statement

$$ \sum_{i=1}^nX_i\xrightarrow{d}N(n\mu,n\sigma^2) $$

would capture this intuition, but doesn't make sense because the "$\xrightarrow{d}$" is a claim in the limit as $n\rightarrow\infty$, and it doesn't make sense to talk about a Gaussian with infinite mean and variance.

Is there a theorem capturing the notion that the distribution of an uncentered and unnormalized sum still approaches a Gaussian? Or is this simply a corollary of the CLT? I'm looking for a proof of something like the following statement:

For a given $\delta > 0$, there exists an $N>0$ such that

$$ \text{distance}\left(\sum_i^nX_i, N(n\mu,n\sigma^2)\right)<\delta $$

for $n>N$ and some distance function of the distributions. That is, if we sum enough random variables, we can get as close to a Gaussian as we like.

$\endgroup$
  • $\begingroup$ The un-normalized sum goes to infinity and so as you mention the distribution has mean and variance growing without bound. I also think you mean some distance function been the distribution of the un-normalized sum and the N(n$\mu$, n$\sigma^2$). $\endgroup$ – Michael R. Chernick May 24 '17 at 23:03
  • 2
    $\begingroup$ Yes: the name of this theorem is the CLT. The point is that if you don't standardize, there is no limiting distribution at all; and if you want there to be a limiting distribution, then you have to change the location and scale in a way that's asymptotically equivalent to standardization. These are all part of the content of the CLT. I emphasized these points in my account of the CLT at stats.stackexchange.com/a/3904/919. $\endgroup$ – whuber May 24 '17 at 23:43
  • $\begingroup$ @MichaelChernick Yes, I mean some distance function and have edited the question to reflect that. I'm open to suggestions if there better way to notate this. $\endgroup$ – fragapanagos May 24 '17 at 23:44
  • $\begingroup$ If you want to talk about behavior in finite samples, you'd need to go to something like the Berry-Esséen inequality. While the inequality is stated with the variables in standardized form, the bound on the difference in cdf isn't affected by the horizontal scaling factor nor by a shift. This isn't specifically convergence because we're dealing with some particular $n$ but it may do what you need for a particular sense of "close to Gaussian" (N.B. a small bound on the difference in cdf doesn't imply Gaussian-like behavior) $\endgroup$ – Glen_b -Reinstate Monica May 25 '17 at 0:25
  • $\begingroup$ Thanks @Glen_b. The Berry-Esséen inequality seems to supply what I was searching for. $\endgroup$ – fragapanagos May 25 '17 at 0:48
0
$\begingroup$

If you want to talk about behavior in finite samples, you'd need to go to something like the Berry-Esséen inequality.

While the inequality is stated with the variables in standardized form, the bound on the difference in cdf isn't affected by the horizontal scaling factor nor by a shift.

This isn't specifically convergence because we're dealing with some particular $n$ but it may do what you need for a particular sense of "close to Gaussian"

Note, however, that a close-to-Gaussian cdf implied by a small bound on the difference in cdf doesn't imply Gaussian-like behavior of the variable. For example, it's possible for a variable that has at most a miniscule absolute deviation from a Gaussian (e.g.bounded by some fixed but small $\epsilon >0$) to have infinite variance and no mean.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.