How close to reality is the normal distribtion?
The height of people is supposed to be normally distributed but the probaility of a 30 foot person in a normal distribution is not zero. Does it matter?
14
-
$\begingroup$ I think that's what the parameters (mean, stdev) determine. For example a low stdev (imposing a very skinny PDF) may impose a smaller likelihood of extreme values. Please others, correct me if I'm wrong. $\endgroup$– RaynorCommented May 1, 2012 at 2:26
-
$\begingroup$ Re: "How close to reality is the normal distribtion?" - what reality are you referring to? The answer to that question is entirely context dependent - the distribution of what? It's hard to tell what is being asked here. $\endgroup$– MacroCommented May 1, 2012 at 2:38
-
$\begingroup$ The mathematician Henri Poincare is alleged to have said "Everybody believes in the normal distribution, the physicists because they think it is a mathematical theorem and the mathematicians because they believe it is a physical law". It is not known what beliefs, if any, he ascribed to statisticians. $\endgroup$– Dilip SarwateCommented May 1, 2012 at 2:39
-
2$\begingroup$ Are you aware that a Normal approximation to the distribution of heights of people (using a mean of, say, 66" and a standard deviation of 3") gives the probability of a 30 foot (or taller) person as around $10^{-2088}$? That's an excellent correspondence with reality! $\endgroup$– whuber ♦Commented May 1, 2012 at 6:37
-
3$\begingroup$ @Nesp Since facts are facts, do you ever use $(4\pi/3)r^3$ as the volume of an actual sphere (say a ball bearing) of radius $r$ and ever worry that the surface is not perfectly smooth as the surface of a mathematical sphere must be, and that the division into arbitrarily small pieces that is needed for the derivation of $(4\pi/3)r^3$ is inapplicable to an actual sphere that exists in real life as opposed to the mathematical world? Or is $(4\pi/3)r^3$ close enough not just for gummint purposes but for yours as well? $\endgroup$– Dilip SarwateCommented May 1, 2012 at 13:25
|
Show 9 more comments
2 Answers
$\begingroup$
$\endgroup$
4
This is a very interesting question, indeed!
The fact that a given distribution violates some physical limits, is why I love Bayes' Theorem so much. It is true that the normal distribution performs very well in a lot of situations, but (if I understood your question correctly) what annoys you is that it actually violates some physical constraints (e.g., that a person can't be taller than 30 feet). The question is, then, how do we correct for this? Is it possible to take into account this BEFORE we do the measurement? The answer is yes: we can account for this.
What you are actually trying to accomplish is to put some a-priori certain limits to a distribution, and this can be included via te a-priori distribution. For example, if we want to incorporate this limit for the height of a person, we could do it via the a-posteriori distribution of heights $h$, i.e., using Bayes' theorem,
$$p(h|D,I)=\frac{p(D|h,I)p(h|I)}{p(D|I)}.$$
Here $D$ is the data, $h$ is the parameter (the height in this case) and $I$ is any a-priori information that you have concerning the distribution of the parameter $h$. As you may recall, $p(D|h,I)$ is the likelihood, where you actually define the sampling distribution of your data (maybe you are measuring persons' height assuming gaussian errors). On the other hand, $p(h|I)$ is the a-priori distribution of the parameter $h$ and is exactly what you are looking for: here you impose some constraints on the value of $h$. For example, if you know that a person by physical constraints cannot be smaller than $h_{low}$ and cannot be larger than $h_{high}$, then this enters via the a-priori distribution: $$p(h|I)=f(h),\ \ \ h_{low}<h<h_{high}$$ and is $0$ otherwise. However, the real problem is how to define a "good" a-priori distribution. You might think of the uniform distribution, i.e., assume $$p(h|I)=\frac{1}{h_{high}-h_{low}},\ \ \ h_{low}<h<h_{high}$$ but this can affect in a serious way your distribution because not all ranges of values in a logaritmic scale are equal (this means that the probability of obtaining, e.g., $1<h<10$ is lower than obtaining $10<h<100$ and, therefore, the distribution is scale dependent!). When you measure a scale parameter (like height or width), a more reliable a-priori distribution is the Jeffrey's prior, i.e., $$p(h|I)=\frac{1}{h\ln(h_{high}/h_{min})},\ \ \ h_{low}<h<h_{high}$$
-
$\begingroup$ Can you explain the relevance of this discussion to the original question? $\endgroup$– MacroCommented May 1, 2012 at 2:48
-
$\begingroup$ I edited it :-). I realized that the link wasn't clear. $\endgroup$– NéstorCommented May 1, 2012 at 2:48
-
$\begingroup$ Bayes theorem brings up a whole set of other questions as you have to make assumptions about the high and the low. If you don't have data for the entire population, wouldn't it be unsafe to make any hard assumptions about what the limits are? $\endgroup$– mfcCommented May 1, 2012 at 3:09
-
$\begingroup$ It is in general, but I believe it is not in the case of the heights. There are pretty well established physical limits for the high and low limits of $h$ of a human being (i.e. how high can a human being be without stressing the body). In any case, you can always put extreme physical values (i.e. a human can't be smaller than an atom and can't be higher than the radius of the Earth), and the result would still be pretty much a gaussian. It all goes down to your knowledge on the physical constraints on your problem. $\endgroup$– NéstorCommented May 1, 2012 at 3:24
$\begingroup$
$\endgroup$
The interesting and useful property of the Normal is that it's what we get, very generally, when we average lots of measurements. Thus, the height of a randomly-selected individual isn't Normal, as you note - and their blood pressure, shoe size and number of years of education are also non-Normal - but when we take an average of a sample of these measurements, on different people, their average behaves very like a Normal - and more Normal when we average more measurements. In this sense, yes, the Normal is very like reality, in all sorts of applications.
Moreover, it's not just the average that looks Normal, all sorts of manipulations of averages look Normal too - and this is fundamentally what makes regression work. In fact for many statistical purposes, Normality of data is basically irrelevant; what matters is that it's what you get when averaging lots of similarly-behaved variables. It is a tremendously useful property.
