69
$\begingroup$

A standardized Gaussian distribution on $\mathbb{R}$ can be defined by giving explicitly its density: $$ \frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$

or its characteristic function.

As recalled in this question it is also the only distribution for which the sample mean and variance are independent.

What are other surprising alternative characterization of Gaussian measures that you know ? I will accept the most surprising answer

$\endgroup$
0

13 Answers 13

48
$\begingroup$

My personal most surprising is the one about the sample mean and variance, but here is another (maybe) surprising characterization: if $X$ and $Y$ are IID with finite variance with $X+Y$ and $X-Y$ independent, then $X$ and $Y$ are normal.

Intuitively, we can usually identify when variables are not independent with a scatterplot. So imagine a scatterplot of $(X,Y)$ pairs that looks independent. Now rotate by 45 degrees and look again: if it still looks independent, then the $X$ and $Y$ coordinates individually must be normal (this is all speaking loosely, of course).

To see why the intuitive bit works, take a look at

$$ \left[ \begin{array}{cc} \cos45^{\circ} & -\sin45^{\circ} \newline \sin45^{\circ} & \cos45^{\circ} \end{array} \right] \left[ \begin{array}{c} x \newline y \end{array} \right]= \frac{1}{\sqrt{2}} \left[ \begin{array}{c} x-y \newline x+y \end{array} \right] $$

$\endgroup$
9
  • 5
    $\begingroup$ Jay - this is basically a re-statement of the mean and variance being independent. $X+Y$ is a re-scaled mean and $X-Y$ is a rescaled standard deviation. $\endgroup$ Commented Mar 15, 2011 at 7:24
  • 7
    $\begingroup$ @probabilityislogic - I like the intuition of what you said, but I don't think it is exactly a restatement because $X - Y$ isn't exactly a rescaling of the SD: the SD forgets the sign. So independence of mean and SD follows from independence of $X+Y$, $X-Y$ (when $n = 2$), but not the other way around. That may have been what you meant by "basically". Anyway, it's good stuff. $\endgroup$
    – user1108
    Commented Mar 15, 2011 at 16:20
  • 4
    $\begingroup$ Where can we find proof for this property? $\endgroup$
    – Royi
    Commented Aug 2, 2013 at 14:45
  • 1
    $\begingroup$ @Royi see 16. here. For (a), note that $2X=(X+Y)+(X-Y)$. For (b) note that $\varphi(2t)\varphi(-2t)=(\varphi(t)\varphi(-t))^4$ which yearns for the substitution $\psi(t)=\varphi(t)\varphi(-t)$ from which you get $\psi(t)=\psi^{2^{2n}}(\frac{t}{2^n})$. If $\varphi(t_0)=0$, then $\psi(t_0)=0$, hence for all $n$, $\psi(\frac{t_0}{2^n})=0$ and there is a sequence $t_n$ such that $t_n\to 0$ and $\varphi(t_n)=0$ for all $n$, which contradicts continuity of $\varphi$ at $0$. (c) is straighforward [continued] $\endgroup$ Commented Apr 15, 2017 at 11:38
  • 1
    $\begingroup$ For (d), $\gamma(t)=\gamma^{2^n}(\frac{t}{2^n})$. Note that $\varphi(t)=1-\frac{t^2}2+o(t^2)$, hence $\gamma(t)=1+o(t^2)$. Plug this in the previous equality and prove that for fixed $t$, $\lim_n \gamma^{2^n}(\frac{t}{2^n}) = 1$ which implies $\gamma(t)=1$ for all $t$. This means $\varphi$ is real, and the equality in (a) turns into what is asked. Again, prove that $\varphi(t)=\varphi^{2^{2n}}(\frac{t}{2^n})$ and use $\varphi(t)=1-\frac{t^2}2+o(t^2)$ to get $\lim_n \varphi^{2^{2n}}(\frac{t}{2^n}) = e^{-t^2/2}$. Hence $\varphi(t)=e^{-t^2/2}$ and $X$ is normal. $\endgroup$ Commented Apr 15, 2017 at 11:43
31
$\begingroup$

The continuous distribution with fixed variance which maximizes differential entropy is the Gaussian distribution.

$\endgroup$
26
$\begingroup$

There's an entire book written about this: "Characterizations of the normal probability law", A. M. Mathai & G. Perderzoli. A brief review in JASA (Dec. 1978) mentions the following:

Let $X_1, \ldots, X_n$ be independent random variables. Then $\sum_{i=1}^n{a_i x_i}$ and $\sum_{i=1}^n{b_i x_i}$ are independent, where $a_i b_i \ne 0$, if and only if $X_i$ [are] normally distributed.

$\endgroup$
7
  • 6
    $\begingroup$ there must be a condition such $<a,b>=0$ missing ? for example if n=2 $a_i=b_i=1$ $X_1+X_2$ and $X_1+X_2$ are not independant. $\endgroup$ Commented Dec 17, 2010 at 8:38
  • 2
    $\begingroup$ @robin good catch. I've been puzzling over the implicit quantifiers, too. Unfortunately, all I have access to is that (titillating) quotation from the review, not the book. It would be fun to find it in a library and browse through it... $\endgroup$
    – whuber
    Commented Dec 17, 2010 at 23:03
  • $\begingroup$ This feels like a generalization of G. Jay Kerns' (currently #1) answer. $\endgroup$
    – vqv
    Commented Dec 19, 2010 at 21:53
  • $\begingroup$ I think you may be looking for the Lukacs & King (1954) paper. See this answer at math.SE with a link to the aforementioned paper. $\endgroup$
    – cardinal
    Commented Mar 14, 2012 at 4:00
  • 2
    $\begingroup$ Where this proposition says "where $a_i b_i\ne 0$", does it mean for EVERY set of scalars where $a_i b_i \ne 0$"? I hate seeing "where" used in place of "for every" or "for some". "Where" should be used to explain one's notation, as in "where $c$ is the speed of light and $g$ is the gross domestic product", etc. $\endgroup$ Commented Feb 9, 2013 at 1:23
22
$\begingroup$

Stein’s Lemma provides a very useful characterization. $Z$ is standard Gaussian iff $$E f’(Z) = E Z f(Z)$$ for all absolutely continuous functions $f$ with $E|f’(Z)| < \infty$.

$\endgroup$
0
20
$\begingroup$

Gaussian distributions are the only sum-stable distributions with finite variance.

$\endgroup$
4
  • 8
    $\begingroup$ That they are sum stable and that they are the unique ones with finite variance are both forced on us by the CLT. The interesting part of this assertion is that there exist other sum-stable distributions! $\endgroup$
    – whuber
    Commented Nov 9, 2010 at 23:12
  • 1
    $\begingroup$ @whuber: indeed! this characterization is a bit contorted, and the other sum-stable distributions are perhaps more curious. $\endgroup$
    – shabbychef
    Commented Nov 10, 2010 at 16:58
  • $\begingroup$ @whuber actually, I do not see how the CLT implies this fact. It only seems to tell us that asymptotically, the sum of normals is normal, not that any finite sum is normally distributed. Or do you have to somehow use Slutsky's theorem as well? $\endgroup$
    – shabbychef
    Commented Nov 11, 2010 at 4:20
  • 3
    $\begingroup$ Adopting the usual standardization, a sum of two normals is the sum of one normal distribution X_0 plus the limiting distribution of a series X_1, X_2, ..., whence the sum is the limiting distribution of X_0, X_1, ..., which by the Lindeberg-Levy CLT is normal. $\endgroup$
    – whuber
    Commented Nov 11, 2010 at 13:28
14
$\begingroup$

Theorem [Herschel-Maxwell]: Let $Z \in \mathbb{R}^n$ be a random vector for which (i) projections into orthogonal subspaces are independent and (ii) the distribution of $Z$ depends only on the length $\|Z\|$. Then $Z$ is normally distributed.

Cited by George Cobb in Teaching statistics: Some important tensions (Chilean J. Statistics Vol. 2, No. 1, April 2011) at p. 54.

Cobb uses this characterization as a starting point for deriving the $\chi^2$, $t$, and $F$ distributions, without using Calculus (or much probability theory).

$\endgroup$
10
$\begingroup$

This is not a characterization but a conjecture, which dates back from 1917 and is due to Cantelli:

If $f$ is a positive function on $\mathbb{R}$ and $X$ and $Y$ are $N(0,1)$ independent random variables such that $X+f(X)Y$ is normal, then $f$ is a constant almost everywhere.

Mentioned by Gérard Letac here.

$\endgroup$
6
  • $\begingroup$ it is good that you mention it! I can't figure out the intuition, do you? $\endgroup$ Commented Feb 15, 2011 at 17:51
  • 2
    $\begingroup$ If Gérard Letac has not managed to prove it, it could stay an open conjecture for quite a while...! $\endgroup$
    – Xi'an
    Commented Jan 12, 2012 at 20:31
  • 1
    $\begingroup$ @Xi'an: I fully agree, of course. (Didn't know you were roaming in these quarters of the web... Good news that you are.) $\endgroup$
    – Did
    Commented Jan 12, 2012 at 21:14
  • 1
    $\begingroup$ @DidierPiau: thanks!, I just saw this announcement: <a href="mathnet.ru/php/… counterexample to Cantelli's conjecture</a> given by Victor Kleptsyn at the Dobrushin Mathematics Laboratory Seminar a few weeks ago. But no abstract and no trace of a preprint anywhere. $\endgroup$
    – Xi'an
    Commented Jan 12, 2012 at 21:25
  • 8
    $\begingroup$ @Xi'an Here is a preprint by Victor Kleptsyn and Aline Kurtzmann with a counterexample to the Cantelli conjecture. The construction uses a new tool, which the authors call the Brownian mass transport, and yields a discontinuous function $f$. The authors state that they believe that Cantelli conjecture holds if one asks that $f$ is continuous (theirs is a mixture of two continuous functions). $\endgroup$
    – Did
    Commented Apr 26, 2012 at 8:11
10
$\begingroup$

Suppose one is estimating a location parameter using i.i.d. data $\{x_1,...,x_n\}$. If $\bar{x}$ is the maximum likelihood estimator, then the sampling distribution is Gaussian. According to Jaynes's Probability Theory: The Logic of Science pp. 202-4, this was how Gauss originally derived it.

$\endgroup$
2
  • $\begingroup$ I'm not sure I understand this as a characterisation of the normal distribution, so I'm probably missing something. What if we had iid Poisson data and wanted to estimate $\mu$? The MLE is $\bar{x}$ but the sampling distribution of $\bar{x}$ isn't Gaussian - firstly, $\bar{x}$ has to be rational; secondly, if it were Gaussian, so would be $\sum{x_i}$ but that's $\text{Poisson}(n\mu)$. $\endgroup$
    – Silverfish
    Commented Dec 17, 2014 at 13:05
  • 4
    $\begingroup$ The Poisson mean is not a location parameter! $\endgroup$ Commented Apr 5, 2015 at 18:58
9
$\begingroup$

Let $\eta$ and $\xi$ be two independent random variables with a common symmetric distribution such that

$$ P\left ( \left |\frac{\xi+\eta}{\sqrt{2}}\right | \geq t \right )\leq P(|\xi|\geq t).$$

Then these random variables are gaussian. (Obviously, if the $\xi$ and $\eta$ are centered gaussian, it is true.)

This is the Bobkov-Houdre Theorem

$\endgroup$
7
$\begingroup$

A more particular characterisation of the normal distribution among the class of infinitely divisible distributions is presented in Steutel and Van Harn (2004).

A non-degenerate infinitely divisible random variable $X$ has a normal distribution if and only if it satisfies $$-\limsup_{x\rightarrow\infty}\dfrac{\log{\mathbb P}(\vert X\vert>x)}{x\log(x)}=\infty.$$

This result characterises the normal distribution in terms of its tail behaviour.

$\endgroup$
2
  • 1
    $\begingroup$ A short proof of the stated limit goes as follows: If $X$ is standard normal, then $x \mathbb P(X>x)/\varphi(x)\to 1$ as $x\to \infty$, so $\log\mathbb P(X>x)−\log \varphi(x) + \log x \to 0$. But $2 \log \varphi(x) \sim −x^2$ and so the result follows. A rough sketch for the case of the Poisson seems to indicate that the given limit is $\lambda$, but I didn't check that too closely. $\endgroup$
    – cardinal
    Commented Jun 6, 2012 at 19:41
  • $\begingroup$ This seems a bit of a cheat to me, because the initial restriction to infinitely divisible distributions is a very strong one. Thus, the Normal distribution is singled out as lying on the boundary of a relatively tiny space: interesting but not surprising. $\endgroup$
    – whuber
    Commented Jan 17, 2020 at 14:34
6
$\begingroup$

In the context of image smoothing (e.g. scale space), the Gaussian is the only rotationally symmetric separable* kernel.

That is, if we require $$F[x,y]=f[x]f[y]$$ where $[x,y]=r[\cos\theta,\sin\theta]$, then rotational symmetry requires \begin{align} F_\theta &= f'[x]f[y]x_\theta+f[x]f'[y]y_\theta \\ &= -f'[x]f[y]y+f[x]f'[y]x = 0 \\ &\implies \\ \frac{f'[x]}{xf[x]} &= \frac{f'[y]}{yf[y]} = \mathrm{const.} \end{align} which is equivalent to $\log\big[f[x]\big]'=cx$.

Requiring that $f[x]$ be a proper kernel then requires the constant be negative and the initial value positive, yielding the Gaussian kernel.


*In the context of probability distributions, separable means independent, while in the context of image filtering it allows the 2D convolution to be reduced computationally to two 1D convolutions.

$\endgroup$
4
  • 2
    $\begingroup$ +1 But doesn't this follow from an immediate application of the Herschel-Maxwell theorem in 2D? $\endgroup$
    – whuber
    Commented Jan 10, 2017 at 16:54
  • $\begingroup$ @whuber Indeed, somehow I managed to overlook your answer when looking through this thread! $\endgroup$
    – amoeba
    Commented Jan 11, 2017 at 17:02
  • $\begingroup$ @whuber Yes. I had not read through this old thread in detail, and was just adding this answer by request. $\endgroup$
    – GeoMatt22
    Commented Jan 12, 2017 at 17:02
  • 1
    $\begingroup$ @amoeba see also here. $\endgroup$
    – GeoMatt22
    Commented Jan 12, 2017 at 17:08
3
$\begingroup$

Its characteristic function has the same form as its pdf. I am not sure of another distribution which does that.

$\endgroup$
1
  • 4
    $\begingroup$ See this answer of mine for ways of constructing random variables whose characteristic functions are the same as their pdfs. $\endgroup$ Commented Jul 31, 2012 at 1:23
3
$\begingroup$

Recently Ejsmont [1] published article with new characterization of Gaussian:

Let $(X_1,\dots, X_m,Y) \textrm{ and } (X_{m+1},\dots,X_n,Z)$ be independent random vectors with all moments, where $X_i$ are nondegenerate, and let statistic $\sum_{i=1}^na_iX_i+Y+Z$ have a distribution which depends only on $\sum_{i=1}^n a_i^2$, where $a_i\in \mathbb{R}$ and $1\leq m < n$. Then $X_i $ are independent and have the same normal distribution with zero means and $cov(X_i,Y)=cov(X_i,Z)=0$ for $i\in\{1,\dots,n\}$.

[1]. Ejsmont, Wiktor. "A characterization of the normal distribution by the independence of a pair of random vectors." Statistics & Probability Letters 114 (2016): 1-5.

$\endgroup$
1
  • 1
    $\begingroup$ That's a delicate and fascinating characterization. Thank you for improving this thread by sharing it! $\endgroup$
    – whuber
    Commented Nov 4, 2016 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.