What is the importance of the function $e^{-x^2}$ in statistics?

Question

In my calculus class , we encountered the function $e^{-x^2}$, or the "bell curve", and I was told that it has frequent applications in statistics.

Out of curiosity, I want to ask: Is the function $e^{-x^2}$ truly important in statistics? If so, what is it about $e^{-x^2}$ that makes it useful, and what are some of its applications?

I couldn't find much info about the function on the internet, but after doing some research, I found a link between bell curves in general, and something called normal distribution. A Wikipedia page links these types of functions to statistics application, with highlighting by me, that states:

"The normal distribution is considered the most prominent probability distribution in statistics. There are several reasons for this:1 First, the normal distribution arises from the central limit theorem, which states that under mild conditions the sum of a large number of random variables drawn from the same distribution is distributed approximately normally, irrespective of the form of the original distribution."

So, if I gather a large amount of data from some kind of survey or the like, they could be distributed equally among a function like $e^{-x^2}$? The function is symmetrical, so is its symmetry i.e its usefulness to normal distribution, what makes it so useful in statistics? I'm merely speculating.

In general, what does make $e^{-x^2}$ useful in statistics? If normal distribution is the only area, then what makes $e^{-x^2}$ unique or specifically useful among other gaussian type functions in normal distribution?

Answer 1

The reason that this function is important is indeed the normal distribution and its closely linked companion, the central limit theorem (we have some good explanations of the CLT in other questions here).

In statistics, the CLT can typically be used to compute probabilites approximately, making statements like "we are 95 % confident that..." possible (the meaning of "95 % confident" is often misunderstood, but that's a different matter).

The function $\exp\Big(-\frac{(x-\mu)^2}{2\sigma^2}\Big)$ is (a scaled version of) the density function of the normal distribution. If a random quantity can be modelled using the normal distribution, this function describes how likely different possible values of said quantity are. Outcomes in regions with high density are more likely than outcomes in regions with low density.

$\mu$ and $\sigma$ are parameters that determine the location and scale of the density function. It is symmetric about $\mu$, so changing $\mu$ means that you shift the function to the right or to the left. $\sigma$ determines the value of the density function at its maximum ($x=\mu$) and how quickly it goes to 0 as $x$ moves away from $\mu$. In that sense, changing $\sigma$ changes the scale of the function.

For the particular choice $\mu=0$ and $\sigma=1/\sqrt{2}$ the density is (proportional to) $e^{-x^2}$. This is not a particularly interesting choice of these parameters, but it has the benefit of yielding a density function that looks slightly simpler than all others.

On the other hand, we can go from $e^{-x^2}$ to any other normal density by the change-of-variables $x=\frac{u-\mu}{\sqrt{2}\sigma}$. The reason that your textbook says that $e^{-x^2}$, and not $\exp\Big(-\frac{(x-\mu)^2}{2\sigma^2}\Big)$, is a very important function is that $e^{-x^2}$ is simpler to write.

Answer 2

You are right, normal distribution or gaussian is a scaled and shifted $\exp (-x^2)$, so the importance of $\exp (-x^2)$ comes mostly from the fact that it is essentially the normal distribution.

And normal distribution is important mainly because ("under mild regularity conditions") the sum of many independent and identically distributed random variables approaches normal, when "many" approaches infinity.

Not everything is normally distributed. For example, your survey results may not be, at least if the responses are not even on the continuous scale but something like integers 1–5. But the mean of the results is normally distributed over repeated sampling, because the mean is just a scaled (normalized) sum, and the individual responses are independent of each other. Assuming the sample is large enough, of course, because strictly speaking, normality appears only when the size of the sample becomes infinite.

As you see from the example, normal distribution may appear as a result of the estimation or modeling process, even when the data is not normally distributed. Therefore normal distributions are everywhere in statistics. In bayesian statistics, many posterior distributions of parameters are approximately normal, or can be assumed to be.

Answer 3

Unique feature of this function is that its spectrum density (Fourier transform) is the same as a function itself. This means that when it is properly scaled to be the probability density function (PDF), its moment generating function (MGF) and characteristic functions (CF) have the same form and scaling:

• PDF: $\frac 1 {\sqrt{s\pi}}e^{-x^2/2}$, it is Gaussian (normal) PDF
• MGF: $\frac 1 {\sqrt{s\pi}}e^{t^2/2}$
• CF: $\frac 1 {\sqrt{s\pi}}e^{-t^2/2}$

This leads to an endless list of amazing implications. I'll give you my favorite: uncertainty principle in quantum mechanics.

As you may have heard in quantum mechanics the particle are represented by their wave functions $\psi(x)$. The square of the amplitude of the wave function $|\psi(x)|^2$ is basically a PDF, i.e. represents the probability of the particle being at the vicinity of $x$. Thus quantum mechanics is very intimately linked to theory of probability.

You may also have heard that in quantum mechanics it is impossible to detect both the location $x$ of a particle and its speed (momentum) $p$ exactly. If you know exactly where the particle is, you won't be able to determine it speed. If you know the speed at which a particle is moving, you won't know where it is. If you know spectral analysis, then you should know that a perfect sine wave is not local, it spans from $-\infty<x<\infty$. It's the same principle here.

This is captured in uncertainty principle, and sometimes it is approximately expressed as $\Delta x\Delta p\ge \frac{\hbar}{2}$. Here's the punchline: the quantity $\Delta x\Delta p$ is the smallest when the wave function of the particle $\psi(x)$ is represented by normal distribution PDF, i.e. your function! No wonder that it is also the probability distribution with maximum entropy among all with variance 1.

So, just trying to explain you one example I had to reference several fundamental concepts in probability theory and statistics PDF, CF, spectral analysis, entropy etc. That is because no matter what are you doing in statistics this function is lurking somewhere near by.

Answer 4

One version of CLT tells us that the distribution of averages of independent identically distributed random variables will start to look like the bell-shaped normal distribution as the number of variables in the sum ($n$) gets large. Formal mathematical convergence takes place under mild conditions on the distribution when the average is appropriately normalized. This will work for most population distributions of various shapes including gamma, triangular, uniform, beta, chi square and even discrete distributions like Bernoulli. This makes it easy to do inference on the mean of a distribution based on a random sample by testing hypotheses or constructing confidence intervals based on the approximating normal distribution. Because the variance of the sample mean goes to $0$ at a rate of $1/n$, the mean will actually converge to a degenerate distribution with all its probability mass at the population mean. So the appropriate normalization for convergence to a normal requires recentering and multiplication by $\sqrt{n}$. There are other statistics that come up that also converge to the normal. The fact that the normal distribution can be used to approximate the distribution of various test statistics is the reason for its prominence in statistics.