1
$\begingroup$

I have some generated data that I want to follow a given Gaussian distribution $N(\mu,\sigma)$, and I would like to quantify the distance between the generated distribution and $N(\mu,\sigma)$.

Specifically, I am generating a set of atomic positions and I want the distribution of the lengths between each atom and its $k$ nearest-neighbors (k is fixed) to be as close as possible to $N(\mu,\sigma)$. For that purpose, I sample $m$ points from $N(\mu,\sigma)$, where $m$ is the number of generated atoms, and I would like to evaluate the distance between the two distributions (generated and sampled from $N$).

The generating process is a deep neural network, which implies that I need this distance to be as smooth (continuous and differentiable) as possible so that the DNN gets the most meaningful gradient information possible.

In this case, it is more important to match the mean and the standard deviation $(\mu,\sigma)$ than to match the actual Gaussian shape. I am a bit lost in the wide range of different measures that are available..

My question

What would be a suitable statistical distance in this case ?

Should I always compare the Cumulative/Empirical distributions functions ? Are there other ways to compare PDFs ?

|cite|improve this question
$\endgroup$
  • $\begingroup$ It's hard to say what would be best for your case without knowing your case. Provide context. $\endgroup$ – Kodiologist Jul 12 '17 at 18:34
  • $\begingroup$ @Kodiologist I've added some clarifications ! $\endgroup$ – Tool Jul 12 '17 at 18:51
  • $\begingroup$ Unless $k$ is very small or close to the entire count of atoms, what you are asking appears geometrically impossible, since the number of nearest neighbors (if there is to be any hope that $\mu$ and $\sigma$ are constant over space) will be, on average, proportional to the cube of the distance. $\endgroup$ – whuber♦ Jul 12 '17 at 21:28
  • $\begingroup$ @whuber I don't understand what you mean. $k$ is indeed very small (from 2 to 10) I don't see the problem of constraining the bond length distribution to be close to a given distribution. Of course it depends on how we create the bonds i.e. how we select the 'nearest' neighbors. $\endgroup$ – Tool Jul 20 '17 at 20:10
0
$\begingroup$

There are quite a few such measures between probability distributions, each with their own properties and goals. For example, you could use earth mover's distance. If these measures don't emphasize the mean and SD enough for your tastes, you can add the absolute distances between the observed and desired mean and the observed and desired SD as penalty terms when evaluating your neural network.

Note that in any case, it makes sense to compare the empirical distribution of your simulated atomic distances to the theoretical normal distribution, rather than using the empirical distribution of a simulated normal distribution.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thank you for these infos ! If I understand well, you suggest that I should compare my empirical distribution with the true continuous distribution, but how would I achieve that without sampling from it ? ( How can I compare discrete and continuous distributions ?) Also, how EMD is better that f-divergences in my case ? $\endgroup$ – Tool Jul 12 '17 at 19:23
  • $\begingroup$ @Tool You can replace the discrete distribution with a piecewise uniform one to get a continuous distribution, then, I believe, compute the earth mover's distance with numerical integration. I don't know of a reason that an $f$-divergence would be worse or better than the earth mover's distance in this case. $\endgroup$ – Kodiologist Jul 12 '17 at 23:04
  • $\begingroup$ Thank you for these clarifications. I think I'll start with $f$-divergences first since they are computationally cheaper ! Do you know of any references that compare the different properties of the different measures ? $\endgroup$ – Tool Jul 14 '17 at 17:37
  • $\begingroup$ @Tool Nope, sorry. $\endgroup$ – Kodiologist Jul 14 '17 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.