I have some generated data that I want to follow a given Gaussian distribution $N(\mu,\sigma)$, and I would like to quantify the distance between the generated distribution and $N(\mu,\sigma)$.
Specifically, I am generating a set of atomic positions and I want the distribution of the lengths between each atom and its $k$ nearest-neighbors (k is fixed) to be as close as possible to $N(\mu,\sigma)$. For that purpose, I sample $m$ points from $N(\mu,\sigma)$, where $m$ is the number of generated atoms, and I would like to evaluate the distance between the two distributions (generated and sampled from $N$).
The generating process is a deep neural network, which implies that I need this distance to be as smooth (continuous and differentiable) as possible so that the DNN gets the most meaningful gradient information possible.
In this case, it is more important to match the mean and the standard deviation $(\mu,\sigma)$ than to match the actual Gaussian shape. I am a bit lost in the wide range of different measures that are available..
My question
What would be a suitable statistical distance in this case ?
Should I always compare the Cumulative/Empirical distributions functions ? Are there other ways to compare PDFs ?