19
$\begingroup$

Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$.

I know this isn't enough for all distributions.

I wish to see some examples of distributions where even with a large sample size (perhaps 100, or 1000, or higher), the distribution of the sample mean is still fairly skewed.

I know I have seen such examples before, but I can't remember where and I can't find them.

$\endgroup$
  • 5
    $\begingroup$ Consider a Gamma distribution with shape parameter $\alpha$. Take the scale as 1 (it doesn't matter). Let's say you regard $\text{Gamma}(\alpha_0,1)$ as just "sufficiently normal". Then a distribution for which you need to get 1000 observations to be sufficiently normal has a $\text{Gamma}(\alpha_0/1000,1)$ distribution. $\endgroup$ – Glen_b Jun 15 '13 at 14:13
  • 1
    $\begingroup$ @Glen_b, why not make that an official answer & develop it a bit? $\endgroup$ – gung Jun 15 '13 at 14:28
  • 4
    $\begingroup$ Any sufficiently contaminated distribution will work, along the same lines as @Glen_b's example. E.g., when the underlying distribution is a mixture of a Normal(0,1) and a Normal(huge value, 1), with the latter having only a tiny probability of appearing, then interesting things happen: (1) most of the time, the contamination does not appear and there is no evidence of skewness; but (2) sometimes the contamination appears and the skewness in the sample is enormous. The distribution of the sample mean will be highly skewed regardless but bootstrapping (e.g.) will usually not detect it. $\endgroup$ – whuber Jun 15 '13 at 17:02
  • 1
    $\begingroup$ @whuber's example is instructive, showing that the central limit theorem can, in theory, be arbitrarily misleading. In practical experiments, I suppose one needs to ask oneself whether there could be some huge effect that occurs very rarely, and apply the theoretical result with a little circumspection. $\endgroup$ – David Epstein Jun 19 '13 at 7:40
19
$\begingroup$

Some books state a sample size of size 30 or higher is necessary for the central limit theorem to give a good approximation for $\bar{X}$.

This common rule of thumb is pretty much completely useless. There are non-normal distributions for which n=2 will do okay and non-normal distributions for which much larger $n$ is insufficient - so without an explicit restriction on the circumstances, the rule is misleading. In any case, even if it were kind of true, the required $n$ would vary depending on what you were doing. Often you get good approximations near the centre of the distribution at small $n$, but need much larger $n$ to get a decent approximation in the tail.

Edit: See the answers to this question for numerous but apparently unanimous opinions on that issue, and some good links. I won't labour the point though, since you already clearly understand it.

I am wanting to see some examples of distributions where even with a large sample size (maybe 100 or 1000 or higher), the distribution of the sample mean is still fairly skewed.

Examples are relatively easy to construct; one easy way is to find an infinitely divisible distribution that is non-normal and divide it up. If you have one that will approach the normal when you average or sum it up, start at the boundary of 'close to normal' and divide it as much as you like. So for example:

Consider a Gamma distribution with shape parameter $α$. Take the scale as 1 (scale doesn't matter). Let's say you regard $\text{Gamma}(α_0,1)$ as just "sufficiently normal". Then a distribution for which you need to get 1000 observations to be sufficiently normal has a $\text{Gamma}(α_0/1000,1)$ distribution.

So if you feel that a Gamma with $\alpha=20$ is just 'normal enough' -

Gamma(20) pdf

Then divide $\alpha=20$ by 1000, to get $\alpha = 0.02$:

Gamma(0.02) pdf

The average of 1000 of those will have the shape of the first pdf (but not its scale).

If you instead choose an infinitely divisible distribution that doesn't approach the normal, like say the Cauchy, then there may be no sample size at which sample means have approximately normal distributions (or, in some cases, they might still approach normality, but you don't have a $\sigma/\sqrt n$ effect for the standard error).

@whuber's point about contaminated distributions is a very good one; it may pay to try some simulation with that case and see how things behave across many such samples.

$\endgroup$
12
$\begingroup$

In addition to the many great answers provided here, Rand Wilcox has published excellent papers on the subject and has shown that our typical checking for adequacy of the normal approximation is quite misleading (and underestimates the sample size needed). He makes an excellent point that the mean can be approximately normal but that is only half the story when we do not know $\sigma$. When $\sigma$ is unknown, we typically use the $t$ distribution for tests and confidence limits. The sample variance may be very, very far from a scaled $\chi^2$ distribution and the resulting $t$ ratio may look nothing like a $t$ distribution when $n=30$. Simply put, non-normality messes up $s^2$ more than it messes up $\bar{X}$.

$\endgroup$
  • 2
    $\begingroup$ This is a good point to make; it's often not actually the mean that people deal with but some function of it and other things. However it's not only $s^2$ that can be messed up - you also lose independence of numerator and denominator, and that can have some surprising effects in the tails. $\endgroup$ – Glen_b Jun 16 '13 at 13:13
9
$\begingroup$

You might find this paper helpful (or at least interesting):

http://www.umass.edu/remp/Papers/Smith&Wells_NERA06.pdf

Researchers at UMass actually carried out a study similar to what you're asking. At what sample size do certain distributed data follow a normal distribution due to CLT? Apparently a lot of data collected for psychology experiments is not anywhere near normally distributed, so the discipline relies pretty heavily on CLT to do any inference on their stats.

First they ran tests on data that was uniform, bimodal, and one distibution that was normal. Using Kolmogorov-Smirnov, the researchers tested how many of the distributions were rejected for normality at the $\alpha = 0.05$ level.

Table 2. Percentage of replications that departed normality based on the KS-test. 
 Sample Size 
           5   10   15   20   25  30 
Normal   100   95   70   65   60  35 
Uniform  100  100  100  100  100  95 
Bimodal  100  100  100   75   85  50

Oddly enough, 65 percent of the normally distributed data were rejected with a sample size of 20, and even with a sample size of 30, 35% were still rejected.

They then tested several several heavily skewed distributions created using Fleishman's power method:

$Y = aX + bX^2 +cX^3 + dX^4$

X represents the value drawn from the normal distribution while a, b, c, and d are constants (note that a=-c).

They ran the tests with sample sizes up to 300

Skew  Kurt   A      B      C       D 
1.75  3.75  -0.399  0.930  0.399  -0.036 
1.50  3.75  -0.221  0.866  0.221   0.027 
1.25  3.75  -0.161  0.819  0.161   0.049 
1.00  3.75  -0.119  0.789  0.119   0.062 

They found that at the highest levels of skew and kurt (1.75 and 3.75) that sample sizes of 300 did not produce sample means that followed a normal distribution.

Unfortunately, I don't think that this is exactly what you're looking for, but I stumbled upon it and found it interesting, and thought you might too.

$\endgroup$
  • 4
    $\begingroup$ "Oddly enough, 65 percent of the normally distributed data were rejected with a sample size of 20, and even with a sample size of 30, 35% were still rejected." -- then it sounds like they're using the test wrong; as a test of normality on completely specified normal data (which is what the test is for), if they're using it right, it must be exact. $\endgroup$ – Glen_b Jun 15 '13 at 8:26
  • 5
    $\begingroup$ @Glen_b: There are multiple sources of potential error here. If you read the document, you'll note that what is listed as "normal" here is actually normal random variates with mean 50 and standard deviation of 10 rounded to the nearest integer. So, in that sense, the test used is already using a misspecified distribution. Second, it still appears they have performed the tests incorrectly, as my attempts at replication show that for a sample mean using 20 such observations, the rejection probability is about 27%. (cont.) $\endgroup$ – cardinal Jun 15 '13 at 14:29
  • 5
    $\begingroup$ (cont.) Third, regardless of the above, some software may use the asymptotic distribution and not the actual one, though at sample sizes of 10K this shouldn't matter too much (if ties had not been artificially induced on the data). Finally, we find the following rather strange statement near the end of that document: Unfortunately, the properties of the KS-test in S-PLUS limit the work. The p-values for the present study were all compiled by hand over the multiple replications. A program is needed to calculate the p-values and make a judgment on them compared to the alpha level chosen. $\endgroup$ – cardinal Jun 15 '13 at 14:32
  • 3
    $\begingroup$ Hi @Glen_b. I don't believe the rounding will reduce the rejection rate here because I believe they were testing against the true standard normal distribution using the rounded data (which is what I meant by saying the test used a misspecified distribution). (Perhaps you were, instead, thinking of using the KS test on a discrete distribution.) The sample size for the KS test was 10000, not 20; they did 20 replications at sample size 10000 each to get the table. At least, that was my understanding of the description from skimming the document. $\endgroup$ – cardinal Jun 15 '13 at 23:02
  • 3
    $\begingroup$ @cardinal - you're correct, of course, so perhaps that could be the source of a substantial chunk of the rejections at large sample sizes. Re: "The sample size for the KS test was 10000, not 20" ... okay, this is sounding increasingly odd. One is left to wonder why they'd think either of those conditions was of much value, rather than say the other way around. $\endgroup$ – Glen_b Jun 15 '13 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.