5
$\begingroup$

I apply statistical methods in sociological researches and now I feel a bit confused as I found out more about CLT.

For instance, if I have sample of 1000 observations, do I even need to check it for normality as CLT states that if sample is big enough, our data approximates normal distribution? And if it holds up, why people constantly plot q-qplot or do tests like shapiro to test for normality? (Plus we know that many of stat methods such as ANOVA, t test quite robust to violation of normality)

Of course, when population is normally distributed and we have big enough sample we can be sure that our sample is normally distributed, but what if it is not, but I still have big sample(e.g. 1000), should I really worry about the fact that my data violates this assumption?

Sorry if this is stupid question. Thank you.

$\endgroup$
1

3 Answers 3

19
$\begingroup$

CLT states that if sample is big enough, our data approximates normal distribution

That is false.

If you take a very large sample from a non-normally distributed population, the empirical distribution of the sample has high probability of being close to that of the population, which is not normal. It does not in any way converge to the normal distribution.

CLT says that the sampling distribution of the sample mean, or of the sample sum, approaches the normal distribution as the sample size grows. That is quite a different matter.

As for $1000$ being a big enough sample to get a good approximation to the normal distribution, that is not true of very skewed population distributions. For example, if $X_1,X_2,X_3,\ldots\sim\operatorname{Poisson}(0.001)$ then $X_1+\cdots+X_{1000}\sim\operatorname{Poisson}(1),$ and that is not close to the normal distribution.

$\endgroup$
1
  • 3
    $\begingroup$ A nit, but in light of the misconceptions, potentially an important one: neither of your characterizations of the CLT is correct, because they omit the critical aspect of standardizing the statistic. $\endgroup$
    – whuber
    Nov 15, 2021 at 18:28
9
$\begingroup$

do I even need to check it for normality as CLT states that if sample is big enough, our data approximates normal distribution?

  • It is wrong that, if the sample is big enough the distribution of the data/population approaches a normal distribution.

    Instead, the CLT relates to (the limit of) the mean of samples (or other types of sums of variables).

  • But you are right that the sample distribution of the test statistic, which is used to estimate parameters of the population distribution or estimate the error/variance, will often approach a normal distribution independent from the underlying distribution of the population.

So for large sample sizes the violation of the assumption that the error distribution is a normal distribution becomes less of a problem. (Ironically the test of normality becomes more powerful and likely to reject the hypothesis of the assumption)

$\endgroup$
3
$\begingroup$

As Dave mentioned, a very similar question on this topic was asked about a year ago, which is worth looking over. I happen to be one of the respondents, and you can read my answer here.

The following is going to be in rough terms aimed at making things easily understandable. For more details and technicalities, see my answer (and attached comments) that I linked above.

Brief explanation

The CLT essentially states that the distribution of the mean of the data will become normal as the sample size increases, it says nothing about the distribution of the data itself. So you could have normal, exponential, bimodal, ect. distributed data but, if your sample size is large enough, the distribution of the mean will be normal.

For example, say you are measuring a metric from a population that is exponentially distributed. Imagine you take 100 random samples from the population where each sample is large enough for the CLT to apply. If you were then to take the mean of those 100 samples, the CLT essentially states that those 100 means would be normally distributed. So the repeated measures of the mean are normally distributed, despite the fact the underlying data is actually exponentially distributed.

If this is still a bit confusing, I personally found this (quirky) video very helpful as it does a great job illustrating these concepts.

So why does this all matter?

Knowing the means are normally distributed allows us to use a variety of parametric tests (anova, t-test, ect.) which operate on the assumption that this is true. This is both the power and pitfall of the CLT: the ability to compare non-normally distributed samples to one another.

I think where this gets confusing is that just because you can compare two sample means to each other doesn't mean that you should. (ie comparing the means of two exponential distributions might not tell you what you think it does, or two bi-modal distributions, or a bi-modal with a uni-modal distribution, ect).

The question most people should ask is, "is the mean (or a difference in means) a useful metric given the distribution of my data". Only if the answer to this question is yes, should one proceed to compare means (thus relying on the CLT). This is where those Q-Q plots and tests for normality you mentioned come in. They examine the distribution of the data itself to see if it is normal. This is important as the distribution of the data is critical to keep in mind when interpreting the results of a statistical test. If the data does not follow the distribution you expected it to (in this case normal) that comparison of means likely doesn't tell you what you think it does.

$\endgroup$
7
  • 2
    $\begingroup$ The last parameter is a bit weird "the normality test is used in order to decide whether a comparison of means is meaningful". Whether it is meaningful depends on the problem/application and not so much on the distribution. $\endgroup$ Nov 16, 2021 at 9:10
  • 2
    $\begingroup$ The CLT may not always work with a Pareto distribution. $\endgroup$ Nov 16, 2021 at 9:11
  • 1
    $\begingroup$ @SextusEmpiricus: For illustration see stats.stackexchange.com/questions/192652/… $\endgroup$ Nov 16, 2021 at 12:12
  • $\begingroup$ @SextusEmpiricus Good points, I will edit my answer accordingly. I agree I didn't state the last part very well. It sounds like the normality of the data is often important to the problems/applications in the question asker's field (which often leads to certain default assumptions/intuitions around comparing means) which is what I was trying to get at with "meaningful". $\endgroup$
    – Cole
    Nov 17, 2021 at 9:15
  • 1
    $\begingroup$ @Cole I do like the before-last paragraph "just because you can compare two sample means to each other doesn't mean that you should". $\endgroup$ Nov 17, 2021 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.