1
$\begingroup$

In Christopher Bishop's PRML book,in section 1.4, the author explains how intuition fails in higher dimensions. He does this by using a gaussian distribution in higher dimensions. He concludes the following-

If we transform from Cartesian to polar coordinates, and then integrate out the directional variables, we obtain an expression for the density p(r) as a function of radius r from the origin. Thus p(r)δr is the probability mass inside a thin shell of thickness δr located at radius r. This distribution is plotted, for various values of D, in Figure 1.23, and we see that for large D the probability mass of the Gaussian is concentrated in a thin shell. enter image description here

Exercise 1.20 I think, is supposed to have proven this. It first takes an equation that supposedly describes the gaussian distribution in higher dimensions, like so-

$$p(\textbf{x})=\frac{1}{(2\pi\sigma^2)^{D/2}}\exp\left(-\frac{\Vert\textbf{x}\Vert^2}{2\sigma^2}\right)$$

It then converts it into polar coordinates and finds the density w.r.t r, in which the "direction variables have been integrated out".

$$p(r)=\frac{S_Dr^{D-1}}{(2\pi\sigma^2)^{D/2}}\exp\left(-\frac{r^2}{2\sigma^2}\right)$$

My questions

My questions are the following-

  1. How does the author obtain the first equation? Isn't the equation that describes a higher dimensional gaussian distribution the following?

$$N(\textbf{x}|\mu,\Sigma)=\frac{1}{(2\pi)^{D/2}}\frac{1}{|\Sigma|^{1/2}}\exp\left(-\frac{1}{2}(\textbf{x}-\mu)^T\Sigma^{-1}(\textbf{x}-\mu)\right)$$

I think he considers the mean to be 0 for every rv, but why isn't the covariance matrix there in the equation?

  1. How do we get the second equation, the one in the polar coordinate system, from the first? And what does the author mean by "integrating out the directional variables"?

For more context, Exercise 1.20

The question

enter image description here

The book's solution

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

When $x\sim\mathcal N_p(0,\sigma^2I_p)$, its density is $$\varphi(x)=\frac{1}{(2\pi\sigma^2)^{D/2}}\,\exp\left(-\frac{\sigma^{-2}}{2}\textbf{x}^T\textbf{x}\right)=\frac{1}{(2\pi\sigma^2)^{D/2}}\,\exp\left(-\frac{\vert\vert \mathbf x\vert\vert^2}{2\sigma^{2}}\right)$$ The change to Cartesian $\mathbf x=(x_1,\ldots,x_p)$ from spherical coordinates $(\varrho,\theta_1,\ldots,\theta_{p-1})$ is given by $$ \mathbf x=\left(\begin{matrix} \varrho\sin\theta_1\\ \varrho\cos\theta_1\sin\theta_2\\ \quad\vdots\\ \varrho\cos\theta_1\cdots\cos\theta_{p-1} \end{matrix}\right) $$ and its Jacobian is $$\varrho^{p-1}\prod_{i=1}^{p-1} \sin^{p-2}\theta_i$$ Hence $$p(\varrho,\boldsymbol{\theta})= \exp\{-\varrho^2/2\sigma^2\}\,\varrho^{p-1}\prod_{i=1}^{p-1} \sin^{p-i-1}\theta_i$$ and $$p(\varrho)= \exp\{-\varrho^2/2\sigma^2\}\,\varrho^{p-1}\int\prod_{i=1}^{p-1} \sin^{p-i-1}\theta_i\,\text d\boldsymbol{\theta}$$

$\endgroup$
2
  • $\begingroup$ I fear I don't understand this. Is there any material you can recommend so I understand this? Which area of mathematics discuss these? $\endgroup$ Commented May 30, 2023 at 3:17
  • $\begingroup$ This involves differential calculus and multivariate integration. I cannot recommend a textbook as it depends on your background and country. The link to the Wikipedia page I put in the answer should help a lot. $\endgroup$
    – Xi'an
    Commented May 30, 2023 at 5:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.