53
$\begingroup$

A standardized Gaussian distribution on $\mathbb{R}$ can be defined by giving explicitly its density: $$ \frac{1}{\sqrt{2\pi}}e^{-x^2/2}$$

or its characteristic function.

As recalled in this question it is also the only distribution for which the sample mean and variance are independent.

What are other surprising alternative characterization of Gaussian measures that you know ? I will accept the most surprising answer

$\endgroup$

14 Answers 14

39
$\begingroup$

My personal most surprising is the one about the sample mean and variance, but here is another (maybe) surprising characterization: if $X$ and $Y$ are IID with finite variance with $X+Y$ and $X-Y$ independent, then $X$ and $Y$ are normal.

Intuitively, we can usually identify when variables are not independent with a scatterplot. So imagine a scatterplot of $(X,Y)$ pairs that looks independent. Now rotate by 45 degrees and look again: if it still looks independent, then the $X$ and $Y$ coordinates individually must be normal (this is all speaking loosely, of course).

To see why the intuitive bit works, take a look at

$$ \left[ \begin{array}{cc} \cos45^{\circ} & -\sin45^{\circ} \newline \sin45^{\circ} & \cos45^{\circ} \end{array} \right] \left[ \begin{array}{c} x \newline y \end{array} \right]= \frac{1}{\sqrt{2}} \left[ \begin{array}{c} x-y \newline x+y \end{array} \right] $$

$\endgroup$
  • 3
    $\begingroup$ Jay - this is basically a re-statement of the mean and variance being independent. $X+Y$ is a re-scaled mean and $X-Y$ is a rescaled standard deviation. $\endgroup$ – probabilityislogic Mar 15 '11 at 7:24
  • 5
    $\begingroup$ @probabilityislogic - I like the intuition of what you said, but I don't think it is exactly a restatement because $X - Y$ isn't exactly a rescaling of the SD: the SD forgets the sign. So independence of mean and SD follows from independence of $X+Y$, $X-Y$ (when $n = 2$), but not the other way around. That may have been what you meant by "basically". Anyway, it's good stuff. $\endgroup$ – user1108 Mar 15 '11 at 16:20
  • 4
    $\begingroup$ Where can we find proof for this property? $\endgroup$ – Royi Aug 2 '13 at 14:45
  • 1
    $\begingroup$ @Royi see 16. here. For (a), note that $2X=(X+Y)+(X-Y)$. For (b) note that $\varphi(2t)\varphi(-2t)=(\varphi(t)\varphi(-t))^4$ which yearns for the substitution $\psi(t)=\varphi(t)\varphi(-t)$ from which you get $\psi(t)=\psi^{2^{2n}}(\frac{t}{2^n})$. If $\varphi(t_0)=0$, then $\psi(t_0)=0$, hence for all $n$, $\psi(\frac{t_0}{2^n})=0$ and there is a sequence $t_n$ such that $t_n\to 0$ and $\varphi(t_n)=0$ for all $n$, which contradicts continuity of $\varphi$ at $0$. (c) is straighforward [continued] $\endgroup$ – Gabriel Romon Apr 15 '17 at 11:38
  • 1
    $\begingroup$ For (d), $\gamma(t)=\gamma^{2^n}(\frac{t}{2^n})$. Note that $\varphi(t)=1-\frac{t^2}2+o(t^2)$, hence $\gamma(t)=1+o(t^2)$. Plug this in the previous equality and prove that for fixed $t$, $\lim_n \gamma^{2^n}(\frac{t}{2^n}) = 1$ which implies $\gamma(t)=1$ for all $t$. This means $\varphi$ is real, and the equality in (a) turns into what is asked. Again, prove that $\varphi(t)=\varphi^{2^{2n}}(\frac{t}{2^n})$ and use $\varphi(t)=1-\frac{t^2}2+o(t^2)$ to get $\lim_n \varphi^{2^{2n}}(\frac{t}{2^n}) = e^{-t^2/2}$. Hence $\varphi(t)=e^{-t^2/2}$ and $X$ is normal. $\endgroup$ – Gabriel Romon Apr 15 '17 at 11:43
27
$\begingroup$

The continuous distribution with fixed variance which maximizes differential entropy is the Gaussian distribution.

$\endgroup$
22
$\begingroup$

There's an entire book written about this: "Characterizations of the normal probability law", A. M. Mathai & G. Perderzoli. A brief review in JASA (Dec. 1978) mentions the following:

Let $X_1, \ldots, X_n$ be independent random variables. Then $\sum_{i=1}^n{a_i x_i}$ and $\sum_{i=1}^n{b_i x_i}$ are independent, where $a_i b_i \ne 0$, if and only if $X_i$ [are] normally distributed.

$\endgroup$
  • 3
    $\begingroup$ there must be a condition such $<a,b>=0$ missing ? for example if n=2 $a_i=b_i=1$ $X_1+X_2$ and $X_1+X_2$ are not independant. $\endgroup$ – robin girard Dec 17 '10 at 8:38
  • 1
    $\begingroup$ @robin good catch. I've been puzzling over the implicit quantifiers, too. Unfortunately, all I have access to is that (titillating) quotation from the review, not the book. It would be fun to find it in a library and browse through it... $\endgroup$ – whuber Dec 17 '10 at 23:03
  • $\begingroup$ This feels like a generalization of G. Jay Kerns' (currently #1) answer. $\endgroup$ – vqv Dec 19 '10 at 21:53
  • $\begingroup$ I think you may be looking for the Lukacs & King (1954) paper. See this answer at math.SE with a link to the aforementioned paper. $\endgroup$ – cardinal Mar 14 '12 at 4:00
  • 2
    $\begingroup$ Where this proposition says "where $a_i b_i\ne 0$", does it mean for EVERY set of scalars where $a_i b_i \ne 0$"? I hate seeing "where" used in place of "for every" or "for some". "Where" should be used to explain one's notation, as in "where $c$ is the speed of light and $g$ is the gross domestic product", etc. $\endgroup$ – Michael Hardy Feb 9 '13 at 1:23
17
$\begingroup$

Gaussian distributions are the only sum-stable distributions with finite variance.

$\endgroup$
  • 8
    $\begingroup$ That they are sum stable and that they are the unique ones with finite variance are both forced on us by the CLT. The interesting part of this assertion is that there exist other sum-stable distributions! $\endgroup$ – whuber Nov 9 '10 at 23:12
  • 1
    $\begingroup$ @whuber: indeed! this characterization is a bit contorted, and the other sum-stable distributions are perhaps more curious. $\endgroup$ – shabbychef Nov 10 '10 at 16:58
  • $\begingroup$ @whuber actually, I do not see how the CLT implies this fact. It only seems to tell us that asymptotically, the sum of normals is normal, not that any finite sum is normally distributed. Or do you have to somehow use Slutsky's theorem as well? $\endgroup$ – shabbychef Nov 11 '10 at 4:20
  • 3
    $\begingroup$ Adopting the usual standardization, a sum of two normals is the sum of one normal distribution X_0 plus the limiting distribution of a series X_1, X_2, ..., whence the sum is the limiting distribution of X_0, X_1, ..., which by the Lindeberg-Levy CLT is normal. $\endgroup$ – whuber Nov 11 '10 at 13:28
17
$\begingroup$

Stein’s Lemma provides a very useful characterization. $Z$ is standard Gaussian iff $$E f’(Z) = E Z f(Z)$$ for all absolutely continuous functions $f$ with $E|f’(Z)| < \infty$.

$\endgroup$
12
$\begingroup$

Theorem [Herschel-Maxwell]: Let $Z \in \mathbb{R}^n$ be a random vector for which (i) projections into orthogonal subspaces are independent and (ii) the distribution of $Z$ depends only on the length $\|Z\|$. Then $Z$ is normally distributed.

Cited by George Cobb in Teaching statistics: Some important tensions (Chilean J. Statistics Vol. 2, No. 1, April 2011) at p. 54.

Cobb uses this characterization as a starting point for deriving the $\chi^2$, $t$, and $F$ distributions, without using Calculus (or much probability theory).

$\endgroup$
9
$\begingroup$

Let $\eta$ and $\xi$ be two independent random variables with a common symmetric distribution such that

$$ P\left ( \left |\frac{\xi+\eta}{\sqrt{2}}\right | \geq t \right )\leq P(|\xi|\geq t).$$

Then these random variables are gaussian. (Obviously, if the $\xi$ and $\eta$ are centered gaussian, it is true.)

This is the Bobkov-Houdre Theorem

$\endgroup$
9
$\begingroup$

This is not a characterization but a conjecture, which dates back from 1917 and is due to Cantelli:

If $f$ is a positive function on $\mathbb{R}$ and $X$ and $Y$ are $N(0,1)$ independent random variables such that $X+f(X)Y$ is normal, then $f$ is a constant almost everywhere.

Mentioned by Gérard Letac here.

$\endgroup$
  • $\begingroup$ it is good that you mention it! I can't figure out the intuition, do you? $\endgroup$ – robin girard Feb 15 '11 at 17:51
  • 2
    $\begingroup$ If Gérard Letac has not managed to prove it, it could stay an open conjecture for quite a while...! $\endgroup$ – Xi'an Jan 12 '12 at 20:31
  • 1
    $\begingroup$ @Xi'an: I fully agree, of course. (Didn't know you were roaming in these quarters of the web... Good news that you are.) $\endgroup$ – Did Jan 12 '12 at 21:14
  • 1
    $\begingroup$ @DidierPiau: thanks!, I just saw this announcement: <a href="mathnet.ru/php/… counterexample to Cantelli's conjecture</a> given by Victor Kleptsyn at the Dobrushin Mathematics Laboratory Seminar a few weeks ago. But no abstract and no trace of a preprint anywhere. $\endgroup$ – Xi'an Jan 12 '12 at 21:25
  • 6
    $\begingroup$ @Xi'an Here is a preprint by Victor Kleptsyn and Aline Kurtzmann with a counterexample to the Cantelli conjecture. The construction uses a new tool, which the authors call the Brownian mass transport, and yields a discontinuous function $f$. The authors state that they believe that Cantelli conjecture holds if one asks that $f$ is continuous (theirs is a mixture of two continuous functions). $\endgroup$ – Did Apr 26 '12 at 8:11
8
$\begingroup$

Suppose one is estimating a location parameter using i.i.d. data $\{x_1,...,x_n\}$. If $\bar{x}$ is the maximum likelihood estimator, then the sampling distribution is Gaussian. According to Jaynes's Probability Theory: The Logic of Science pp. 202-4, this was how Gauss originally derived it.

$\endgroup$
  • $\begingroup$ I'm not sure I understand this as a characterisation of the normal distribution, so I'm probably missing something. What if we had iid Poisson data and wanted to estimate $\mu$? The MLE is $\bar{x}$ but the sampling distribution of $\bar{x}$ isn't Gaussian - firstly, $\bar{x}$ has to be rational; secondly, if it were Gaussian, so would be $\sum{x_i}$ but that's $\text{Poisson}(n\mu)$. $\endgroup$ – Silverfish Dec 17 '14 at 13:05
  • 2
    $\begingroup$ The Poisson mean is not a location parameter! $\endgroup$ – kjetil b halvorsen Apr 5 '15 at 18:58
6
$\begingroup$

A more particular characterisation of the normal distribution among the class of infinitely divisible distributions is presented in Steutel and Van Harn (2004).

A non-degenerate infinitely divisible random variable $X$ has a normal distribution if and only if it satisfies $$-\limsup_{x\rightarrow\infty}\dfrac{\log{\mathbb P}(\vert X\vert>x)}{x\log(x)}=\infty.$$

This result characterises the normal distribution in terms of its tail behaviour.

$\endgroup$
  • 1
    $\begingroup$ A short proof of the stated limit goes as follows: If $X$ is standard normal, then $x \mathbb P(X>x)/\varphi(x)\to 1$ as $x\to \infty$, so $\log\mathbb P(X>x)−\log \varphi(x) + \log x \to 0$. But $2 \log \varphi(x) \sim −x^2$ and so the result follows. A rough sketch for the case of the Poisson seems to indicate that the given limit is $\lambda$, but I didn't check that too closely. $\endgroup$ – cardinal Jun 6 '12 at 19:41
6
$\begingroup$

In the context of image smoothing (e.g. scale space), the Gaussian is the only rotationally symmetric separable* kernel.

That is, if we require $$F[x,y]=f[x]f[y]$$ where $[x,y]=r[\cos\theta,\sin\theta]$, then rotational symmetry requires \begin{align} F_\theta &= f'[x]f[y]x_\theta+f[x]f'[y]y_\theta \\ &= -f'[x]f[y]y+f[x]f'[y]x = 0 \\ &\implies \\ \frac{f'[x]}{xf[x]} &= \frac{f'[y]}{yf[y]} = \mathrm{const.} \end{align} which is equivalent to $\log\big[f[x]\big]'=cx$.

Requiring that $f[x]$ be a proper kernel then requires the constant be negative and the initial value positive, yielding the Gaussian kernel.


*In the context of probability distributions, separable means independent, while in the context of image filtering it allows the 2D convolution to be reduced computationally to two 1D convolutions.

$\endgroup$
  • 2
    $\begingroup$ +1 But doesn't this follow from an immediate application of the Herschel-Maxwell theorem in 2D? $\endgroup$ – whuber Jan 10 '17 at 16:54
  • $\begingroup$ @whuber Indeed, somehow I managed to overlook your answer when looking through this thread! $\endgroup$ – amoeba says Reinstate Monica Jan 11 '17 at 17:02
  • $\begingroup$ @whuber Yes. I had not read through this old thread in detail, and was just adding this answer by request. $\endgroup$ – GeoMatt22 Jan 12 '17 at 17:02
  • 1
    $\begingroup$ @amoeba see also here. $\endgroup$ – GeoMatt22 Jan 12 '17 at 17:08
3
$\begingroup$

Recently Ejsmont [1] published article with new characterization of Gaussian:

Let $(X_1,\dots, X_m,Y) \textrm{ and } (X_{m+1},\dots,X_n,Z)$ be independent random vectors with all moments, where $X_i$ are nondegenerate, and let statistic $\sum_{i=1}^na_iX_i+Y+Z$ have a distribution which depends only on $\sum_{i=1}^n a_i^2$, where $a_i\in \mathbb{R}$ and $1\leq m < n$. Then $X_i $ are independent and have the same normal distribution with zero means and $cov(X_i,Y)=cov(X_i,Z)=0$ for $i\in\{1,\dots,n\}$.

[1]. Ejsmont, Wiktor. "A characterization of the normal distribution by the independence of a pair of random vectors." Statistics & Probability Letters 114 (2016): 1-5.

$\endgroup$
  • 1
    $\begingroup$ That's a delicate and fascinating characterization. Thank you for improving this thread by sharing it! $\endgroup$ – whuber Nov 4 '16 at 17:26
1
$\begingroup$

Its characteristic function has the same form as its pdf. I am not sure of another distribution which does that.

$\endgroup$
  • 4
    $\begingroup$ See this answer of mine for ways of constructing random variables whose characteristic functions are the same as their pdfs. $\endgroup$ – Dilip Sarwate Jul 31 '12 at 1:23
-1
$\begingroup$

The expectation plus minus the standard deviation are the saddle points of the function.

$\endgroup$
  • 11
    $\begingroup$ This is a property of the Normal distribution, to be sure, but it does not characterize it, because plenty of other distributions also have this property. $\endgroup$ – whuber Nov 10 '10 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.