3
$\begingroup$

If a "distribution" is constant, then CLT is not going to work, obviously. However, even if it is not a constant, but variance is very small, the distribution of the sums is still not normal. For example:

import numpy as np
from collections import Counter

a = np.zeros(1000)
a[0] = 10

samples = [np.sum(np.random.choice(a, 100)) for _ in xrange(100000)]

Result:

>>> Counter(samples)
Counter({0.0: 90339, 10.0: 9141, 20.0: 500, 30.0: 20}) # not normal

Is there any general property or a test that one can do on a distribution to see whether sample measures yield a normal distribution.

Of course I could do a normality test after the fact, but I am more interested about the property of the distribution.

$\endgroup$
  • 2
    $\begingroup$ In view of the Berry-Esseen theorem, it is essentially determined by the ratio of the $\rho = E|X - \mu|^3$ to $\sigma^3$ where $\sigma^2$ is the variance. More specifically, $|F_n(x) - \Phi(x)| \le n^{-1/2} C \rho \sigma^{-3}$ where $C$ can be taken to be $1/2$ and $F_n(x)$ is the cdf of the standardized sum $n^{-1/2}\sum_i (X_i - \mu) / \sigma$. Essentially the problem in your example is that the skewness is too large, but if you increase the sample size you will get approximate normality anyways. $\endgroup$ – guy Oct 27 '14 at 2:24
  • 1
    $\begingroup$ The central limit theorem doesn't say anything about a particular sample size. The central limit theorem works just fine for your example distribution. $\endgroup$ – Glen_b -Reinstate Monica Oct 27 '14 at 4:33
  • $\begingroup$ @Glen_b his example has $N = 100$, and at that sample size isn't big enough for the CLT to give a useful approximation (hence value of finite-sample results like barry-Esseen). $\endgroup$ – guy Oct 27 '14 at 4:51
  • 4
    $\begingroup$ @guy The point I wanted the OP to understand is that the central limit theorem doesn't say anything about when it's a useful approximation; it's a result about what happens in the limit as ${n\to\infty}$. The fact that at some particular $n$ the distribution isn't normal doesn't imply anything about the CLT 'working'; indeed the central limit theorem doesn't "work" at any sample size. Your discussion of the Berry-Esseen theorem is definitely appropriate; in fact if it was in the form of an answer I'd think it well worth an upvote. $\endgroup$ – Glen_b -Reinstate Monica Oct 27 '14 at 5:10
  • $\begingroup$ Thanks for the comments, if I make my sample size larger 50,000, that is larger than my population size, I see that my distribution of sample sizes is approaching normal. $\endgroup$ – Akavall Oct 27 '14 at 14:22
1
$\begingroup$

As far as I know, it is easier to actually look for non-gaussianity/normality than gaussianity/normality.

As I explained in this other post, there are several ways of doing this, where the most intuitive is the search for higher-order cummulants in your data, because the gaussian distribution is the only one that has a finite number of non-zero cummulants (this is a theorem known as Marcinkiewicz's theorem). However, this is not recommended because it is computationally expensive and, of course, calculating every cummulant is not possible in real life.

One other way of measuring (and therefore testing for) non-gaussianity that has been particularly useful in Independant Component Analysis (an application where you need to measure the degree of non-gaussianity of samples) is negentropy. For an introduction on these measures, see these notes by Hyvärinen on the subject which is an extract of the paper by Hyvärinen & Oja (2000) on Independant Component Analysis. If you are interested, search for his papers on efficient ways of calculating negentropy. Also, in this post there is an actual derivation of Negentropy if you are looking for it.

$\endgroup$
1
$\begingroup$

The assumptions of the Classical Central Limit Theorem:

Let $X_1,X_2,...,X_n$ be a sequence of $n$ independent identically distributed random variables with probability density function $f$, mean $\mu$, and variance $\sigma^2$. Moreover, you need to assume that $0 < \sigma < \infty$ so that you have finite variance and that your random variables are truly random and not constants.

If you have what I have stated then the Central Limit Theorem will hold.

Note: This says nothing about the speed of convergence. Convergence is an asymptotic property (in the infinite limit). Convergence as I know it depends on the class of tails your density function has. Even for something with finite variance like the t-distribution (for proper exponent), convergence is not necessarily quick as compared to the continuous uniform distribution.

$\endgroup$
  • 1
    $\begingroup$ It would be good to include some acknowledgment that a variance of $0$ (which is finite) will not work, either, because this case is specifically pointed out in the question. $\endgroup$ – whuber Oct 27 '14 at 15:05
  • $\begingroup$ @whuber Thanks for the comment. I've made some changes. What do you think? $\endgroup$ – kolonel Oct 27 '14 at 16:47
  • $\begingroup$ It's great! (Alas, I cannot upvote it a second time ...) Incidentally, your last comment is interesting. Although a uniform distribution has shorter tails than a Normal distribution, convergence to Normal is immediate in the latter case! Therefore there's more determining the rate of convergence than just the heaviness of the tails: their shape has something to do with it, too. (As the classical proofs show us, the convergence rate is best expressed in terms of cumulants of the distribution.) $\endgroup$ – whuber Oct 27 '14 at 16:51
  • $\begingroup$ @whuber thanks. Yes, Exactly! the book I learned this from uses cumulants for this exercise. $\endgroup$ – kolonel Oct 27 '14 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.