# Statistical distance of arbitrary multivariate distributions

What meaningful statistical distance measure can be used for computing in a meaningful way a distance between two arbitrary multivariate probability distributions? I am interested in doing this computation for the cases, where both compared distributions are either discrete or continuous and for the case, where one of them is continuous and the other one discrete.

-
Kullback-Leibler divergence measures the information lost by approximating one distribution by another one. It is thus not symmetric and not a true metric (although the sum of the K-L divergence between $P$ and $Q$ and the divergence between $Q$ and $P$ should be a metric, I believe), but it works with multivariate continuous and discrete observations: en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence – Stephan Kolassa Jan 10 at 11:54
I am actually looking for a true metric. – Igor Jan 10 at 12:05
What is your goal ? To assess whether two unknown distributions are the same based on samples ? – Stéphane Laurent Jan 10 at 12:26
I think @StéphaneLaurent (+1) is asking the right question here. Some clarification would be helpful. There are various ways to define metrics on a space of probability measures. The "easiest" is the total-variation metric $d_{\mathrm{TV}} = \sup_{A \in \mathcal F} |\mu(A) - \nu(A)|$. Stephan's suggestion can also be turned into a metric. See this answer and also this one (with apologies for the self-citations). – cardinal Jan 10 at 14:14
@StéphaneLaurent: I have several goals. One of them is the approximation of certain continuous distributions by discrete ones. cardinal: Your cited answers are quite interesting. I might be wrong, but I think it is quite complicated to compute the Jensen-Shannon Divergence if one of the Probability distributions is continuous and the other one is discrete. – Igor Jan 10 at 16:59