2
$\begingroup$

I'm reading the chapter 2 of PRML(Pattern Recognition and Machine Learning) and I don't understand the following statement in the first paragraph in page 80:

The Gaussian distribution will be constant on surfaces in $x$-space for which this quadratic form is constant.

My understanding is that the Gaussian will become an uniform distribution when the quadratic form is constant but I'm not sure. And when will the quadratic form be constant?

The quadratic form is:

$$ \Delta^2=(x-\mu)^T\Sigma^{-1}(x-\mu) $$

And the Multi-variate Gaussian is :

$$ N(x|\mu,\Sigma)=\frac{1}{(2\pi)^{(D/2)}}\frac{1}{|\Sigma|^{1/2}}\exp\lbrace -\frac{1}{2}(x-\mu)^T\Sigma^{-1}(x-\mu) \rbrace $$

$\endgroup$
3
  • 3
    $\begingroup$ Please give references as you would expect to find them in good literature. "PRML" = ??? The fact that many readers here will be able to guess is much less important than the fact that many don't want to have to guess. $\endgroup$ – Nick Cox Apr 7 '15 at 12:36
  • $\begingroup$ @NickCox Thanks for pointing that, I have corrected PRML to Pattern Recognition and Machine Learning $\endgroup$ – ningyuwhut Apr 7 '15 at 14:58
  • 3
    $\begingroup$ Thanks but in good literature I expect to see author(s), publication date and publisher. Please don't expect people to Google for basic details; it's your privilege to ask a question, but your responsibility to make it as useful and as intelligible as possible to all who might want to read it. $\endgroup$ – Nick Cox Apr 7 '15 at 15:03
4
$\begingroup$

You are misunderstanding what the text is saying. A density (multivariate or univariate) is a function that typically takes on different values for each value of the argument: $f(x_1)$ and $f(x_2)$ are usually different numbers. For uniform distributions, say $U(a,b)$, $f(x)$ has the same value $(b-a)^{-1}$ for all $x \in (a,b)$ and the same value $0$ for all $x \notin (a,b)$. A (univariate) Gaussian density $f(x)$ has, in general, different values for different $x$'s with minor exceptions: for each real number $t$, $f(\mu+t)$ has the same value as $f(\mu-t)$ for the simple reason that $$\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac 12 \left(\frac{(\mu+t)-\mu}{\sigma}\right)^2\right) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac 12 \left(\frac{(\mu-t)-\mu}{\sigma}\right)^2\right)$$ both quantities being equal to $\frac{1}{\sigma\sqrt{2\pi}}\exp\left(-\frac{t^2}{2\sigma^2}\right)$. Writing all of the above gobbledygook can be avoided by noting that all we are looking for are the points where the quadratic function $(x-\mu)^2$ has the same value, and it is easy to determine that $x_1 = \mu+t$ and $x_2 = \mu-t$ give rise to the same value $t^2$ for $(x-\mu)^2$. For a bivariate Gaussian distribution, the density has constant value wherever a quadratic in two variables has the same value: this quadratic corresponds to an ellipse with center $(\mu_X,\mu_Y)$ in the plane. More generally, the quadratic in $n \geq 2$ variables is the quadratic form $(x-\mu)^T\Sigma^{-1}(x-\mu)$ that you quote, and the set of points where this function has the same value is a surface in $n$-dimensional space.

The Gaussian distribution never becomes a uniform distribution in the usual sense of the word. A Gaussian density function can have the same numerical value for different values of the argument, but that is not the same as saying that the Gaussian random variable (or random vector) is uniformly distributed on those points.

One could say that conditioned on $X$ being in the set of points where the quadratic has fixed value, $X$ is uniformly distributed in that set, but this conditional distribution is not Gaussian in any sense of the word. In the univariate case, the conditional distribution is a discrete distribution: conditionally, $X$ takes on values $\mu+t$ and $\mu-t$ with equal probability, in the bivariate case, the conditional distribution of $(X,Y)$ is uniform on an ellipse but it cannot be described by a joint density. While we can write something like $$f_{X,Y\mid (X,Y)\in A}(x,y\mid A) = \begin{cases}c, & (x,y) \in A,\\0, &\text{otherwise} \end{cases}$$ where $A$ is the ellipse in question, that $f$ is not a joint density: its integral over the plane is $0$ instead of $1$, and Gaussianity cannot be averred in any sense of the word. For example, the conditional marginal distributions of $X$ and $Y$ are not Gaussian distributions.

$\endgroup$
2
  • $\begingroup$ Thanks, I Understand a little bit.Do you mean the $\Delta^2$can't be constant for all different points except for the points on the ellipse in the bivariate Gaussian for example? $\endgroup$ – ningyuwhut Apr 8 '15 at 2:19
  • 1
    $\begingroup$ I don't understand what you are asking. Pick any number $z$, say $5.1$. There is a set of vectors $x$ such that $\Delta^2=(x-\mu)^T\Sigma^{-1}(x-\mu)$ has value $5.1$ for all $x$ in the set. In two dimensions, this set of vectors (or points in the plane) is an ellipse. For all $(x,y)$ on this ellipse, $\Delta^2$ has value $5.1$. For any point that is not on this ellipse $\Delta^2$ does not have value $5.1$; it has a different value, say $6.2$. This other point that you have chosen is on a different ellipse, the one corresponding to having chosen $z=6.2$ to start with, etc. $\endgroup$ – Dilip Sarwate Apr 8 '15 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.