4
$\begingroup$

There is this article where the author Michael Betancourt uses this image to convey the concept of the typical set in a distribution.

enter image description here

I would like to plot the typical set of a univariate or a bivariate Gaussian distribution. Thus, I should multiply the normal density by the $dq$.

But how do I get $dq$?. In 2D, for instance, should I use the derivative of the (circle) surface ($2 \pi q$)? But this would be $dV/dq$, and not just $dq$... right?

I get the intuition (the mass around a point is the product of the density times the volume around that point) but, as you can, see I'm a bit confused on how to compute this differential element.

$\endgroup$
  • 2
    $\begingroup$ You seem to be differentiating too many times. In 2D, the relevant "volume" of a region of radius $r$ is $\pi r^2$, whence its differential volume is $\mathrm{d}(\pi r^2)=2\pi r \mathrm{d} r$. You ought to recognize this as proportional to the area element in polar coordinates. A similar calculation in $n$ dimensions gives a volume element proportional to $r^{n-1}\mathrm{d} r$. Compare that function (at least qualitatively) to the graph of "$dq$" in the figure. $\endgroup$ – whuber Aug 15 '17 at 16:19
6
$\begingroup$

One of the confusing things about concentration of measure is that we're trying to demonstrate deviations away from our naive, low-dimensional intuition. Here that is demonstrated in how the radial volume changes relative to a uniform distribution over radii. As we move away from a given point the shells of constant radial distance grow bigger and bigger, hence we get exponentially more differential volume as we move out to larger radii contrary to what we would expect from uniform volume growth. More specifically, the volume grows with the $N-1$ power of the radius as discussed in https://en.wikipedia.org/wiki/N-sphere#Spherical_coordinates.

The plot actually comes from the analytical results for a $10$-dimensional independent identically distributed unit Gaussian distribution, as discussed in Section 4.2 of https://github.com/betanalpha/stan_intro.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.