# How well does the normal distribution perform?

How close to reality is the normal distribtion?

The height of people is supposed to be normally distributed but the probaility of a 30 foot person in a normal distribution is not zero. Does it matter?

-
I think that's what the parameters (mean, stdev) determine. For example a low stdev (imposing a very skinny PDF) may impose a smaller likelihood of extreme values. Please others, correct me if I'm wrong. –  C. Pieters May 1 '12 at 2:26
Re: "How close to reality is the normal distribtion?" - what reality are you referring to? The answer to that question is entirely context dependent - the distribution of what? It's hard to tell what is being asked here. –  Macro May 1 '12 at 2:38
The mathematician Henri Poincare is alleged to have said "Everybody believes in the normal distribution, the physicists because they think it is a mathematical theorem and the mathematicians because they believe it is a physical law". It is not known what beliefs, if any, he ascribed to statisticians. –  Dilip Sarwate May 1 '12 at 2:39
Are you aware that a Normal approximation to the distribution of heights of people (using a mean of, say, 66" and a standard deviation of 3") gives the probability of a 30 foot (or taller) person as around $10^{-2088}$? That's an excellent correspondence with reality! –  whuber May 1 '12 at 6:37
@Nesp Since facts are facts, do you ever use $(4\pi/3)r^3$ as the volume of an actual sphere (say a ball bearing) of radius $r$ and ever worry that the surface is not perfectly smooth as the surface of a mathematical sphere must be, and that the division into arbitrarily small pieces that is needed for the derivation of $(4\pi/3)r^3$ is inapplicable to an actual sphere that exists in real life as opposed to the mathematical world? Or is $(4\pi/3)r^3$ close enough not just for gummint purposes but for yours as well? –  Dilip Sarwate May 1 '12 at 13:25

This is a very interesting question, indeed!

The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it actually violates some physical constraints (e.g., that a person can't be taller than 30 feet). The question is, then, how do we correct for this? Is it possible to take into account this BEFORE we do the measurement? The answer is yes: we can account for this.

What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,

$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$

Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$

-
Can you explain the relevance of this discussion to the original question? –  Macro May 1 '12 at 2:48
I edited it :-). I realized that the link wasn't clear. –  Néstor May 1 '12 at 2:48
Bayes theorem brings up a whole set of other questions as you have to make assumptions about the high and the low. If you don't have data for the entire population, wouldn't it be unsafe to make any hard assumptions about what the limits are? –  mfc May 1 '12 at 3:09
It is in general, but I believe it is not in the case of the heights. There are pretty well established physical limits for the high and low limits of $h$ of a human being (i.e. how high can a human being be without stressing the body). In any case, you can always put extreme physical values (i.e. a human can't be smaller than an atom and can't be higher than the radius of the Earth), and the result would still be pretty much a gaussian. It all goes down to your knowledge on the physical constraints on your problem. –  Néstor May 1 '12 at 3:24