Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

What meaningful statistical distance measure can be used for computing in a meaningful way a distance between two arbitrary multivariate probability distributions? I am interested in doing this computation for the cases, where both compared distributions are either discrete or continuous and for the case, where one of them is continuous and the other one discrete.

share|improve this question
2  
Kullback-Leibler divergence measures the information lost by approximating one distribution by another one. It is thus not symmetric and not a true metric (although the sum of the K-L divergence between $P$ and $Q$ and the divergence between $Q$ and $P$ should be a metric, I believe), but it works with multivariate continuous and discrete observations: en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence – Stephan Kolassa Jan 10 at 11:54
I am actually looking for a true metric. – Igor Jan 10 at 12:05
4  
What is your goal ? To assess whether two unknown distributions are the same based on samples ? – Stéphane Laurent Jan 10 at 12:26
1  
I think @StéphaneLaurent (+1) is asking the right question here. Some clarification would be helpful. There are various ways to define metrics on a space of probability measures. The "easiest" is the total-variation metric $d_{\mathrm{TV}} = \sup_{A \in \mathcal F} |\mu(A) - \nu(A)|$. Stephan's suggestion can also be turned into a metric. See this answer and also this one (with apologies for the self-citations). – cardinal Jan 10 at 14:14
@StéphaneLaurent: I have several goals. One of them is the approximation of certain continuous distributions by discrete ones. cardinal: Your cited answers are quite interesting. I might be wrong, but I think it is quite complicated to compute the Jensen-Shannon Divergence if one of the Probability distributions is continuous and the other one is discrete. – Igor Jan 10 at 16:59
show 5 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.