I think this is a fascinating topic and I do not fully understand it. What law of physics makes so that so many natural phenomena have normal distribution? It would seem more intuitive that they would have uniform distribution.
It is so hard for me to understand this and I feel I am missing some information. Can somebody help me with a good explanation or link me to a book/video/article?
Let me start by denying the premise. Robert Geary probably didn't overstate the case when he said (in 1947) "...normality is a myth; there never was, and never will be, a normal distribution." --
the normal distribution is a model*, an approximation that is sometimes more-or-less useful.
$\:$*(about which, see George Box, though I prefer his "...how wrong does it have to be to not be useful" version).
That some phenomena are approximately normal may be no vast surprise, since sums of independent [or even not-too-strongly-correlated effects] should, if there a lot of them and none has a variance that is substantial compared to the variance of the sum of the rest that we might see the distribution tend to look more normal.
The central limit theorem (which is about the convergence to a normal distribution of a standardized sample mean as $n$ goes to infinity under some mild conditions) at least suggests that we might see a tendency toward that normality with sufficiently large but finite sample sizes.
Of course if standardized means are approximately normal, standardized sums will be; this is the reason for the "sum of many effects" reasoning. So if there are a lot of little contributions to the variation, and they're not highly correlated, you might tend to see it.
The Berry-Esseen theorem gives us a statement about it (convergence toward normal distributions) actually happening with standardized sample means for iid data (under slightly more stringent conditions than for the CLT, since it requires that the third absolute moment be finite), as well as telling us about how rapidly it happens. Subsequent versions of the theorem deal with non-identically distributed components in the sum, though the upper bounds on the deviation from normality are less tight.
Less formally, the behavior of convolutions with reasonably nice distributions gives us additional (though closely related) reasons to suspect it might tend to be a fair approximation in finite samples in many cases. Convolution acts as a kind of "smearing" operator that people who use kernel density estimation across a variety of kernels will be familiar with; once you standardize the result (so the variance remains constant each time you do such an operation), there's clear a progression toward increasingly symmetric hill shapes as you repeatedly smooth (and it doesn't much matter if you change the kernel each time).
Terry Tao gives some nice discussion of versions of the Central limit theorem and the Berry-Esseen theorem here, and along the way mentions an approach to a non-independent version of Berry-Esseen.
So there's at least one class of situations where we might expect to see it, and formal reasons to think it really will tend to happen in those situations. However, at best any sense that the result of "sums of many effects" will be normal is an approximation. In many cases it's quite a reasonable approximation (and in additional cases even though the approximation of the distribution isn't close, some procedures that assume normality aren't especially sensitive to the distribution of the individual values, at least in large samples).
There are many other circumstances where effects don't "add" and there we may expect other things to happen; for example, in a lot of financial data effects tend to be multiplicative (effects will move amounts in percentage terms, like interest and inflation and exchange rates for example). There we don't expect normality, but we might sometimes observe a rough approximation to normality on the log scale. In other situations neither can be appropriate, even in a rough sense. For example, inter-event times are generally not going to be well approximated by either normality or normality of logs; there's no "sums" nor "products" of effects to argue for here. There are numerous other phenomena that we can make some argument for a particular kind of "law" in particular circumstances, such as the limiting distributions in extreme values (Fisher-Tippett-Gnedenko or Pickands-Balkema-de Haan theorems, for example).
There is a famous saying by Gabriel Lippmann (physicist, Nobel laureate), as told by Poincaré:
[The normal distribution] cannot be obtained by rigorous deductions. Several of its putative proofs are awful [...]. Nonetheless, everyone believes it, as M. Lippmann told me one day, because experimenters imagine it to be a mathematical theorem, while mathematicians imagine it to be an experimental fact.
-- Henri Poincaré, Le calcul des Probabilités. 1896
[Cette loi] ne s’obtient pas par des déductions rigoureuses; plus d’une démonstration qu’on a voulu en donner est grossière [...]. Tout le monde y croit cependant, me disait un jour M. Lippmann, car les expérimentateurs s’imaginent que c’est un théorème de mathématiques, et les mathématiciens que c’est un fait expérimental.
It seems that we don't have this quote in our List of Statistical Quotes thread, that's why I thought it would be good to post it here.
What law of physics makes so that so many natural phenomena have normal distribution? It would seem more intuitive that they would have uniform distribution.
The normal distribution is a common place in natural sciences. The usual explanation is why it happens in measurement errors is through some form of large numbers or central limit theorem (CLT) reasoning, which usually goes like this: "since the experiment outcomes are impacted by infinitely large number of disturbances coming from unrelated sources CLT suggests that the errors would normally distributed". For instance, here's an excerpt from Statistical Methods in Data Analysis by W. J. Metzger:
Most of what we measure is in fact the sum of many r.v.’s. For example, you measure the length of a table with a ruler. The length you measure depends on a lot of small effects: optical parallax, calibration of the ruler, temperature, your shaking hand, etc. A digital meter has electronic noise at various places in its circuitry. Thus, what you measure is not only what you want to measure, but added to it a large number of (hopefully) small contributions. If this number of small contributions is large the C.L.T. tells us that their total sum is Gaussian distributed. This is often the case and is the reason resolution functions are usually Gaussian.
However, as you must know this doesn't mean that every distribution will be normal, of course. For instance, Poisson distribution is as common in physics when dealing with counting processes. In spectroscopy Cauchy (aka Breit Wigner) distribution is used to describe the shape of radiation spectra and so on.
I realized this after writing: all three distributions mentioned so far (Gaussian, Poisson, Cauchy) are stable distributions, with Poisson being discrete stable. Now that I thought about this, it seems an important quality of a distribution that will make it survive aggregations: if you add a bunch of numbers from Poisson, the sum is a Poisson. This may "explain" (in some sense) why it's so ubiquitous.
In unnatural sciences you have to be very careful with applying normal (or any other) distribution for a variety of reasons. Particularly the correlations and dependencies are an issue, because they may break the assumptions of CLT. For instance, in finance it's well known that many series look like normal but have much heavier tails, which is a big issue in risk management.
Finally, there are more solid reasons in natural sciences for having normal distribution than sort of "hand waving" reasoning that I cited earlier. Consider, Brownian motion. If the shocks are truly independent and infinitesimal, then inevitably the distribution of an observable path will have normal distribution due to CLT, see e.g. Eq.(10) in Einstein's famous work "INVESTIGATIONS ON THE THEORY OF THE BROWNIAN MOVEMENT". He didn't even bother to call it by its today's name "Gaussian" or "normal".
Another example is quantum mechanics. It so happens that if the uncertainty of a coordinate $\Delta x$ and moment $\Delta p$ are from normal distributions, then the total uncertainty $\Delta x \Delta p$ reaches the minimum, Heisenberg's uncertainty threshold, see Eq.235-237 here.
Hence, don't be surprised to get very different reactions to Gaussian distribution use from researchers in different fields. In some fields such as physics, certain phenomena are expected to be linked naturally to Gaussian distribution based on very solid theory backed by enormous amount of observations. In other fields, Normal distribution is used for its technical convenience, handy mathematical properties or other questionable reasons.
there is an awful lot of overly complicated explanations here...
A good way it was related to me is the following:
Roll a single die, and you have an equal likelihood of rolling each number (1-6), and hence, the PDF is constant.
Roll two dice and sum the results together, and the PDF is no longer constant. This is because there are 36 combinations, and the summative range is 2 to 12. The likelihood of a 2 is unique singular combination of 1 + 1 . The likelihood of a 12, is also unique in that it can only occur in a single combination of a 6 + 6. Now, looking at 7, there are multiple combinations, i.e. 3 + 4, 5 + 2, and 6 + 1 (and their reverse permutations). As you work away from the mid-value (i.e. 7), there are lesser combinations for 6 & 8 etc until you arrive at the singular combinations of 2 and 12. This example does not result in a clear normal distribution, but the more die you add, and the more samples you take, then the result will tend towards a normal distribution.
Therefore, as you sum a range of independent variables subject to random variation (which each can have their own PDFs), the more the resulting output will tend to normality. This in Six Sigma terms give us what we call the 'Voice of the Process'. This is what we call the result of 'common-cause variation' of a system, and hence, if the output is tending towards normality, then we call this system 'in statistical process control'. Where the output is non-normal (skewed or shifted), then we say the system is subject to 'special cause variation' in which there has been some 'signal' that has biased the outcome in some way.
Hope that helps.
What law of physics makes so that so many natural phenomena have normal distribution?
No idea. On the other hand I've also no idea whether it's true, or indeed what 'so many' means.
However, rearranging the problem a little, there is good reason to assume (that is, to model) a continuous quantity that you believe to have a fixed mean and variance with a Normal distribution. That's because the Normal distribution is the result of maximizing entropy subject to those moment constraints. Since, roughly speaking, entropy is a measure of uncertainty, that makes the Normal the most non-commital or maximally uncertain choice of distributional form.
Now, the idea that one should choose a distribution by maximizing its entropy subject to known constraints really does have some physics backing in terms of the number of possible ways to fulfill them. Jaynes on statistical mechanics is the standard reference here.
Note that while maximum entropy motivates Normal distributions in this case, different sorts of constraints can be shown to lead to different distributional families, e.g. the familiar exponential, poisson, binomial, etc.
Sivia and Skilling 2005 ch.5 has an intuitive discussion.
As others have pointed out it's due to the central limit theorem which directly implies that the sum of a large number of independent random variables will follow a normal distribution. I will extend this discussion by mentioning some examples of how this applies and another perspective to answer the question.
Take for instance the time it takes you to get to work. This time is affected by many random factors (delays) such as the weather, road conditions, public events, construction, traffic, etc. You may find traffic not to be independent of these but the fact is it's likely itself normal by depending on may independent factors so it doesn't matter.
Likewise, human height is normally distributed because it depends on a combination of (likely) independent genetic factors and environmental factors. More precisely, without these variations we'd all be as high but because they exist, the variation in height follows a normal distribution which makes height itself follow a normal distribution. Apply the same argument for time to work above given that delay would follow a normal distribution.
Another way to ask this question, is to consider that unimodal symmetric distributions will usually be well-approxiamted by the normal distribution and that in many cases, there is no reason for the population to have an asymmetric distribution.
For instance, if it's known that the average height of males is $1.7m$ then one can claim that it's equally likely to find someone who is $1.7m+Δm$ high or $1.7m-Δm$ high, simply because there is no major factor that would drive the asymmetry and make one of these more probable than the other, it's rather a combination of many independent factors. Meanwhile, if we look at something such as the time until a mechanical piece wear out then there is a good reason that asymmetry exists: the more time its operating, the more likely it will eventually stop working as its detoriated by friction.
