Why is Gaussian distribution on high dimensional space like a soap bubble In this famous post "Gaussian Distributions are Soap Bubbles" it is claimed that the distribution of the points looks like a soap bubble (where it is less dense in the center and more dense at the edge) instead of a bold of mold where it is more dense in the center. I would expect that it is more dense in the center like it is in two or three dimensional.
From the post, I could not figure out why this is the case. It uses three figures which I could not figure out what they are telling.

Can someone clarify this figures and the main question why it is looking like a soap bubble in high dimensions?
 A: The post you link to concerns the use of the normal distribution in high-dimensional problems.  So, suppose you are working in a space $\mathbb{R}^m$ where the dimension $m$ is large.  Let $\boldsymbol{I}$ be the $m$-dimensional identity matrix and consider a normal random vector:
$$\mathbf{X} \equiv (X_1,...,X_m) \sim \text{N}(\mathbf{0}, \sigma^2 \boldsymbol{I}).$$
A well-known property of this distribution is that a centered and normed normal random vector is uniformly distributed on the unit sphere.  That is, if we let $\mathcal{S}_r^m \equiv \{ \mathbf{x} \in \mathbb{R}^m | \sum x_i^2 = r^2 \}$ denote the $m$-dimensional sphere with radius $r$, then we have:
$$\frac{\mathbf{X}}{||\mathbf{X}||} \sim \text{U}(\mathcal{S}_1^m).$$
It is also well-known that the distribution of the scaled-norm of the random vector is:
$$\frac{||\mathbf{X}||}{\sigma \sqrt{m}} \sim \frac{\chi_m}{\sqrt{m}}.$$
Taking $m \rightarrow \infty$, the right-hand-side convergences in probability to one.  Thus, for large $m$ we have:
$$\mathbf{X} \overset{\text{Approx}}{\sim} \text{U}(\mathcal{S}_{\sigma \sqrt{m}}^m)$$
This shows that when $m$ becomes large, the points from this normal random vector are approximately distributed on the surface of a unit sphere with radius $\sigma \sqrt{m}$.  This is what the linked post is referring to when it notes that "...in high dimensions, Gaussian distributions are practically indistinguishable from uniform distributions on the unit sphere".
A: I can't answer about what the OP's famous post claims, but let us consider the simpler case of uniform distributions on the unit disc: $(X,Y)$ is uniformly distributed on the unit disc (that is, $f(X,Y)(x,y) = \frac 1\pi$ for $x^2+y^2 < 1$.  What is the probability that $(X,Y)$ is closer to the unit circle, that is, closer to the boundary of the unit disc than it is to the origin (center of the circle)? Well, only those points that lie inside the circle of radius $\frac 12$ are at distance $< \frac 12$ from the origin, and so all points outside this smaller circle are at distance $> \frac 12$ from the origin. It is an easy computation to arrive at
$$P\left(\frac 12 < \sqrt{X^2+Y^2} < 1\right) = 1- P\left(0\leq \sqrt{X^2+Y^2} < \frac 12\right) = 1 - \frac 1\pi \cdot \pi\left(\frac 12\right)^2 = \frac 34.$$
A similar calculation for a uniform distribution on the interior of a unit sphere in 3 dimensions (the pdf has value $\frac{3}{4\pi}$ on the interior) gives
\begin{align}
P\left(\frac 12 < \sqrt{X^2+Y^2+Z^2} < 1\right) &= 1- P\left(0\leq \sqrt{X^2+Y^2+Z^2} < \frac 12\right)\\
&= 1 - \frac{3}{4\pi} \cdot \frac{4\pi}{3}\left(\frac 12\right)^3\\
&= \frac 78.
\end{align}
Generalizing to $n > 3$ dimensions and remembering that the volume of an $n$-dimeensional hypersphere or radius $r$ is proportional to $r^n$, we get by very similar calculations that
$$P\left(\frac 12 < \sqrt{\sum_{i=1}^n X_i^2} < 1\right) = \frac{2^n-1}{2^n},$$
that is, most of the probability mass_ lies closer to the surface of the sphere than to the origin.  As a final comment, note that the $X_i$ are NIBNID random variables which acronym stands for Not Independent But Nonetheless Identically Distributed.
Turning to IID standard Gaussian random variables, the joint density is not uniformly distributed but has a very pronounced peak at the origin. But, there is so little volume near the center of a hypersphere as compared to closer to the surface that when we integrate the density over the volume of a hypersphere of small radius $r$ to find $P\left(\sqrt{\sum_{i=1}^n X_i^2} < r\right)$, most of this probability mass is obtained from the small contributions from the periphery (there are so many of them) and very little from the few but larger contributions from the core; that is, most of the probability mass lies closer to the skin of the orange than to the center. But things change as $r$ increases. Since $\sum_{i=1}^n X_i^2$ is a $\chi^2$ random variable with $n$ degrees of freedom (with mean $n$ and variance $2n$), which for large $n$ can be approximated as a Gaussian random variable with the same mean and variance) most of its probability was lies in the range $\left[n-\sqrt{18n},n-\sqrt{18n}\right] = [\mu-3\sigma,\mu+3\sigma]$. Put another way,
the quantity $P\left({\sum_{i=1}^n X_i^2} < r^2\right)$ is close to $0$ for small $r$ (the nearly empty space inside the soap bubble), and then (regarded as a function of $r$) increases very rapidly with $r$ in the close vicinity of $r=\sqrt n$ (this is the thin skin of the bubble where most of the mass is) to almost $1$, and then very slowly to its asymptotic value of $1$ (the nearly empty space outside the bubble).  In short, the soap bubble analogy is very apt for Gaussian distributions; almost all the probability mass of the joint pdf of $n$ standard Gaussian random variables does indeed lie in a very thin shell of radius $\approx \sqrt n$ and there is very little probability mass that is not in the shell -- both the interior and the exterior of the shell is mostly empty as is the case with soap bubbles.
A: I really think that the vision of an empty bubble is misleading.
(tl-dr: Instead of an empty bubble I think is better to say that it resembles to a star with n-vertices where $n\rightarrow\infty$, or some kind of a non empty fractal structure with the length of its border going to $\infty$). But still more dense in the center.
The higher the dimension is, there are more points close to the border of a n-sphere than to its center (easy to see moving from 2D to 3D), but for a multivariate normal distribution: $$P\left( \left\{ \text{n-sphere center in} \left(0,...,0\right) \text{ and } r=\delta \right\}\right ) > P\left( \left\{ \text{n-sphere center in} \left(\delta,...,0\right) \text{ and } r=\delta \right\}\right )$$
Always, for any dimension, for any $\delta$, the probability is higher the closer to the center (for similar areas of course). So: is still more dense in the center, but is has less center (center is never 0 because remember $dimension \rightarrow \infty$ and not $dimension=\infty$).
Yes... if you pick a point at random is more likely to be in the border than in the center, but not because the probability in the center is "empty as a bubble", but just becasue there are more points close to the border.
This already happens in 2D: in a 2D-Sphere of radius=1 "there are" $\pi/4$ points at distance ½ of the center, while there are $3/4*\pi$ points at distance less than ½ of the border.  And actually with a bivariate normal distribution the probability of picking a point next to the border is higher than picking one in the center
(see code below).
Another way to visualize it is comparing a 2D sphere vrs a 2D star, as the star has “More border” the probability of picking a point next the border in a star is higher, but the start is not “empty”.
Instead of an empty bubble It will be better to say: it resembles to a star with n-vertices where $n\rightarrow\infty$, or some kind of a non empty fractal structure with the length of its border going to $\infty$. But still more dense in the center.
PD:
Using r
library(shotGroups)
 
#using 2d multivariate normal distribution

#probability of choosing a point inside the cercle with radius delta=0.5 
inner_cercle_prob <- pmvnEll(r=0.5, sigma=diag(2), mu=c(0,0), e=diag(2), x0=c(0,0))

#probabiliyt of choosing a point inside the cercle of radius=1
full_cercle_prob <- pmvnEll(r=1, sigma=diag(2), mu=c(0,0), e=diag(2), x0=c(0,0))
  
#probability of choosin a point inside the cercle but closer to the border
corona_prob=full_cercle_prob-inner_cercle_prob
 
#probability of choosin a point outside the cercle 
outside_cercle_prob=1-full_cercle_prob
 
outside_cercle_prob
[1] 0.6065307
corona_prob
[1] 0.2759662
inner_cercle_prob
[1] 0.1175031

#but for cercles with same radius, the one closer to the center as higher prob.
pmvnEll(r=0.5, sigma=diag(2), mu=c(0,0), e=diag(2), x0=c(0,0))
[1] 0.1175031
pmvnEll(r=0.5, sigma=diag(2), mu=c(0,0), e=diag(2), x0=c(0.5,0))
[1] 0.1044914

A: This is an old post with some great responses but I'd like to give a different perspective.
Assume we take a sample $x$ from $\mathcal{N}(\vec0, \mathcal{I})$ in $D$ dimensions. If the high-dimensional Gaussian is hollow, then that would mean at least one coordinate of our sample $x$ deviates from the mean. By the CDF of the normal distribution, the chance of $x$'s first coordinate being within one standard deviation is about $~68.2\%$. Now what is the chance that both the first and second coordinates are within one standard deviation? They are independent, so it's $0.682^2$. By extension, the probability that a sample in $D$-dimensional space is within one standard deviation along every axis is $0.68^D$. Naturally, this goes to 0 very quickly.
A: I don't think it is true that "a Gaussian distribution in higher dimensions looks like a soap bubble". But first let's see why, in accordance with some of the very detailed responses above, one might be led to think so.
In Cartesian coordinates in $D$ dimensions, after standardization, the probability density looks like 
$$ p(\vec{X}) ~d\vec{X} \sim V^{-D/2} \exp(-\frac{||\vec{X}||}{2V}) ~d\vec{X}$$
where $V$ is the 1D variance of each variable.
We can rewrite this in spherical coordinates, and use spherical symmetry to integrate over the D-1 dimensional spheres. I'll leave out the factor corresponding to the volume of the D-1 sphere (not the ball which is the "interior" of the sphere). The radial distribution is
$$ p(r)dr \sim V^{-\frac{D}2} r^{D-1} \exp(-\frac{r^2}{2V})~dr = V^{-\frac{D}2} r^{D-2} \exp(-\frac{r^2}{2V}) ~ \frac12 dr^2, ~r > 0 $$
Introducing $z = \frac{r^2}2$, the distribution is:
$$p(z)dz \sim V^{-\frac{D}2} z^{\frac{D}2-1} \exp(-\frac{z}{V})~dz$$
which is just a Gamma Distribution (in $z$). 
Now you can look up https://en.wikipedia.org/wiki/Gamma_distribution or calculate the mean radius (modulo some irrelevant factors and $\pm 1$):
$$<r> \sim D \sqrt{V}$$ 
and the variance of the radius $<(r-<r>)^2>$:
$$var(r) \sim DV$$
which means the standard deviation of $r$
$$SD(r) \sim\sqrt{DV}$$.
The relative SD:
$$ \frac{SD(r)}{<r>} \sim \frac1{\sqrt{D}}$$
which, as the number of dimensions $D$ increases, tends to 0. 
So we think that looks like a bubble. But, the question is, as a distribution, does the radial distribution $\rightarrow \delta(r-r_0)$ in the limit $D\rightarrow\infty$?
Now let's go back to that D-dimensional distribution in Cartesian coordinates. It is peaked at the origin and falls off as your distance from the origin increases. It looks nothing like a bubble. If you had a mass density distributed like this you would not encounter any bubble that you have to pierce nor any "thickening" at $r_0$, in fact the density would continue to increase as you travelled towards the center. It is only when we integrate over the shells of fixed radius and collapse them to a single radial point do we get the relative standard deviation tending to 0 with increased dimensionality.
So, no, it is not true that "a Gaussian distribution in higher dimensions looks like a soap bubble".
