# Distance measure between probability density functions

I have some generated data that I want to follow a given Gaussian distribution $N(\mu,\sigma)$, and I would like to quantify the distance between the generated distribution and $N(\mu,\sigma)$.

Specifically, I am generating a set of atomic positions and I want the distribution of the lengths between each atom and its $k$ nearest-neighbors (k is fixed) to be as close as possible to $N(\mu,\sigma)$. For that purpose, I sample $m$ points from $N(\mu,\sigma)$, where $m$ is the number of generated atoms, and I would like to evaluate the distance between the two distributions (generated and sampled from $N$).

The generating process is a deep neural network, which implies that I need this distance to be as smooth (continuous and differentiable) as possible so that the DNN gets the most meaningful gradient information possible.

In this case, it is more important to match the mean and the standard deviation $(\mu,\sigma)$ than to match the actual Gaussian shape. I am a bit lost in the wide range of different measures that are available..

My question

What would be a suitable statistical distance in this case ?

Should I always compare the Cumulative/Empirical distributions functions ? Are there other ways to compare PDFs ?

• It's hard to say what would be best for your case without knowing your case. Provide context. – Kodiologist Jul 12 '17 at 18:34
• @Kodiologist I've added some clarifications ! – Tool Jul 12 '17 at 18:51
• Unless $k$ is very small or close to the entire count of atoms, what you are asking appears geometrically impossible, since the number of nearest neighbors (if there is to be any hope that $\mu$ and $\sigma$ are constant over space) will be, on average, proportional to the cube of the distance. – whuber Jul 12 '17 at 21:28
• @whuber I don't understand what you mean. $k$ is indeed very small (from 2 to 10) I don't see the problem of constraining the bond length distribution to be close to a given distribution. Of course it depends on how we create the bonds i.e. how we select the 'nearest' neighbors. – Tool Jul 20 '17 at 20:10

• @Tool You can replace the discrete distribution with a piecewise uniform one to get a continuous distribution, then, I believe, compute the earth mover's distance with numerical integration. I don't know of a reason that an $f$-divergence would be worse or better than the earth mover's distance in this case. – Kodiologist Jul 12 '17 at 23:04
• Thank you for these clarifications. I think I'll start with $f$-divergences first since they are computationally cheaper ! Do you know of any references that compare the different properties of the different measures ? – Tool Jul 14 '17 at 17:37