7
$\begingroup$

Why did the normal distribution become such a popular (important) distribution? I know one reason is because of CLT. Can you please give more reasons?

$\endgroup$
10
$\begingroup$

The main reason that the normal distribution is so popular is because it works (is at least good enough in many situations). The reason that it works is really because of the Central Limit Theorem. Rather than trying to look beyond the CLT, I think you (and others) should better appreciate the CLT (I have a cross-stitch of the CLT hanging on my wall as I type).

We usually teach and think about the CLT in terms of a sample mean (and that is a powerful use of the CLT), but it extends much further than that. The CLT also means that any variable that we measure that is the result of combining many effects (many relative to the degree of relationship between the different pieces) will be approximately normal.

For example: a person's height is determined by many small effects including genetics (there will be several genes that contribute to height), nutrition (not just good/bad, but what was actually eaten each day that the person was growing), environmental polutions (again each day contributed a small effect), and other things. So heights (within sex/race combinations) are approximately normal.

Annual rainfall for a specific area is the summation of the daily rainfall for the year and while the daily rainfall is probably very far from normal (zero inflated) when you add all those days together you get something much more normal.

Binomial distributions are just sums of Bernoullis and a Poisson distribution can be the sum of smaller Poissons, it should not be a surprise that either can be approximated by a normal (if enough pieces are added together).

Most exceptions come when common values are close to a natural boundary (rainfall in the desert, test scores where many students get 100% or close to it, etc.) or when there is a single (or small number) of very strong contributors (height including both sexes or with a spread of ages when kids are still growing). Otherwise there are many things that can be approximated using the normal distribution (and things become even more normal when you average them together from a sample).

So why do we need any more justification than the CLT (not to take away from the other great answers).

dismount soapbox

addition

Since it appears that at least 2 people want to see the cross-stitch (based on comments below) here is a picture:

enter image description here

I also have cross-stitches of Bayes theorem and the mean value theorem of integration, but they are off topic for this question.

$\endgroup$
  • 7
    $\begingroup$ I think we need to see pics of this cross-stitch! $\endgroup$ – Dimitriy V. Masterov Feb 4 '13 at 22:50
10
$\begingroup$

The Wikipedia article on the normal distribution contains many reasons. I'll summarize a few of the more useful ones here - but I really do suggest having a read through that article:

  1. It is entirely characterized by two parameters that are easy to estimate

  2. A sum of two jointly normal random variables is also normal

  3. Uncorrelated, jointly distributed, normal random variables are independent

  4. Normality assumptions frequently result in analytic (as opposed to numeric) solutions to many estimation problems

To this Wikipedia-based list I'll add one more:

  1. Estimators are often consistent when normality is assumed, even if the normality assumption is violated, see eg quasi-maximum likelihood
$\endgroup$
1
$\begingroup$

We all kneel to the central limit theorem.

Here are some slightly less standard reasons why it has become "popular":

  1. Many people never do more than one statistics course or study more than one introductory text. In such courses or texts it is customary to touch on $t$ tests, correlation and regression, for all of which normal distributions should at least be mentioned as context. Conversely, some procedures may be mentioned as not predicated on normal distributions (chi-square or Wilcoxon-Mann-Whitney, etc.), which creates as many problems as it solves. If other distributions are mentioned, the most likely candidates are binomial and Poisson, which fairly clearly apply to different kinds of problems. People who have never studied statistics formally, but nevertheless use it, even for published research, also tend to have a picture of statistics that is similar.

  2. A higher level of understanding entails realising that many named distributions could be relevant or useful, which means learning not just about one or two other distributions but about many more. That is a big jump, requiring more teaching time, and a stronger formal background in mathematics, than is likely for introductory courses. Naturally, there are many exceptions, e.g. physics, engineering and economics students should usually know the right kind of machinery. Unfortunately, many researchers who use statistics, and many non-statisticians who write statistics texts and give courses to people in their own field, work with a foggy kind of myth about statistics, such as that you need normal distributions to do mainstream statistics, except that you can use non-parametric tests instead.

In short, what is popular hinges not just on statistical logic and what works with data, but also on the sociology and psychology of what is taught and remembered and its ugly complement misconceptions of what is central to statistics. At worst, the normal is "popular" because people know about almost nothing else....

$\endgroup$
1
$\begingroup$

I like to view the normal distribution as the curve that approximates (or is the limit of) the sum of many small little random effects. Galton's Bean machines such as the ones in the image below demonstrate this nicely and these simple models make you imagine more easily why and how we see the pattern of the normal curve, or something that looks like it, so often around us.

It also shows why it is not always correct since not always the effect is due to many small effects (or sometimes a few big ones dominate), and also those bean machines are actually binomial distributions, the Gaussian curve (or maybe we should call it the De Moivre curve) is just an approximation to it.

(Yes I know this is just like CLT but it gives that theorem a more practical meaning instead of just being a mathematical theorem, so I'd say this is one of the more reasons. Gauss actually gives another reason, it is the distribution of errors for which the sum of least squares is the maximum likelihood solution.)

https://upload.wikimedia.org/wikipedia/commons/7/7f/Quincunx_%28Galton_Box%29_-_Galton_1889_diagram.png https://commons.wikimedia.org/wiki/File:Quincunx_(Galton_Box)_-_Galton_1889_diagram.png

$\endgroup$
  • $\begingroup$ Even so, this picture requires explanation: very few people really understand or even see the point Galton was trying to make with this machine--and it's not even apparent in these images. $\endgroup$ – whuber Sep 24 '18 at 22:27
  • $\begingroup$ I think that's going to make your post rather obscure and opens it to the possibility of being misunderstood. As I recall, Galton used this version of his Quincunx, with its intermediate level marked by "AA," to provide an intuitive understanding of regression--not for illustrating Normal distributions. The Wikipedia article doesn't even mention this: that's what I mean by "obscure." $\endgroup$ – whuber Sep 24 '18 at 23:04
1
$\begingroup$

Essentially your question is asking about characterisations of the normal distribution. One characterisation of the distribution is that it arises when one takes a second-order Maclaurin approximation to the cumulant generating function of a distribution.


Normal distribution arises by second-order approximation to CGF: Suppose we have a random variable $X$ with (complex) cumulant generating function given by:

$$H(t) = \ln \phi_X(t) = \ln (\mathbb{E}(e^{itX})).$$

This function $H$ is the logarithm of the characteristic function $\phi_X$. It is a convex function with $H(0)=0$ and it can be approximated via a Taylor polynomial. In particular, if we take a second-order Taylor approximation around the value $t=0$ (i.e., a second-order Maclaurin approximation) we get:

$$H(t) \approx K(0) + \frac{H'(0)}{1!} it - \frac{H''(0)}{2!} t^2.$$

To find this approximation we note that the characteristic function has $\phi_X^{(k)}(0) = i^k \mathbb{E}(X^k)$ for all $k \in \mathbb{N}$, which gives us the following derivatives at zero for the cumulant generating function:

$$\begin{equation} \begin{aligned} H(t) &= \ln \phi_X(t) & & \implies & & H(0) = 0, \\[12pt] H'(t) &= \frac{\phi_X'(t)}{\phi_X(t)} & & \implies & & H'(0) = i \mathbb{E}(X), \\[6pt] H''(t) &= \frac{\phi_X''(t)}{\phi_X(t)} - \frac{\phi_X'(t)^2}{\phi_X(t)^2} & & \implies & & H''(0) = - \mathbb{E}(X^2) + \mathbb{E}(X)^2 = - \mathbb{V}(X). \\[6pt] \end{aligned} \end{equation}$$

Substituting these into the second-order Maclaurin approximation we obtain:

$$H(t) \approx \mathbb{E}(X) i t - \frac{\mathbb{V}(X) t^2}{2}.$$

This approximation is the cumulant generating function for the normal distribution. Thus, we see that one characterisation of the normal distribution is that it arises whenever one takes a second-order Maclaurin approximation to the cumulant generating function of a random variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.