Distance Metrics for Comparing Theoretical Sampling Distributions

Question

I have a sampling probability density function (denoted $f(x|M,N)$ that becomes hard to calculate for large degrees of freedom (i.e. as $M$ and $N$ get big). If I have another pdf (say $f^{*}(x|M,N)$ such that,

\begin{equation} \lim_{M\to\infty} \lim_{N\to\infty} f(x|M,N)=f^{*}(x|M,N) \end{equation}

what distance metrics would be best for determining the values of $M$ and $N$ such that $f^{*}$ closely approximates $f$? Part of my problem is that I am not sure how to define "closely approximates"... I have tried calculating the total variation distance between the two pdf's while varying the sample sizes but am unsure if this is a good metric to use. I am not very familiar with distance metrics and would appreciate any thoughts.

Answer 1

Choosing the right metric is not an easy task. It depends on what kind of closeness you want to achieve. Different metrics measure different types of closeness between two distributions. There is a paper that discusses several types of metrics between distributions:

http://onlinelibrary.wiley.com/doi/10.1111/j.1751-5823.2002.tb00178.x/abstract

Preprint: https://www.math.hmc.edu/~su/papers.dir/metrics.pdf