# Is there any probability distance that preserves all properties of a metric?

In studying Kullback–Leibler distance, there are two things we learn very quickly is that it does not respect neither the triangle inequality nor the symmetry, required properties of a metric.

My question is whether there is any metric of probability density functions that fulfil all the constraints of a metric.

• To focus on probability densities is to focus on the "wrong" object. As for metrics, there are the "classical" ones, e.g., Lévy (and the related Ky Fan metric on random variables), Wasserstein along with ones closer in spirit to KL, e.g., Jensen-Shannon divergence. Though mostly overlooked historically, note that in the original KL paper, the KL divergence was indeed symmetric (though still not a metric). Commented Feb 17, 2014 at 17:54
• @cardinal, well, I'm not so much in the field, can you please suggest the "right" object? Commented Feb 17, 2014 at 17:57
• J.C.: Sorry, the comment box became too small for all I was trying to fit in there. I should have elaborated. The cumulative distribution function turns out to be a more general and natural object of study. :-) Commented Feb 18, 2014 at 0:44
• @cardinal why? ;) Commented Feb 18, 2014 at 8:49

Take a look at this paper that covers a wide range of popular metrics on the space of probability measure. My personal favorites are the total variation distance and $$L^2$$ Wasserstein distance (earth mover distance).

• That is a good paper - especially figure 1. I'm saving a copy of that for future reference.
– Pat
Commented Feb 17, 2014 at 17:57
• @Ramanujan updated to an arxiv link. Commented Dec 8, 2021 at 22:11

I believe that the Earth Mover's Distance, also known as the Wasserstein metric, is an example which meets your requirements.

There are some modifications to the KL divergence that make it acquire some of the metric properties (though not all).

For example, the Jeffrey’s divergence modifies the KL divergence to make it symmetric.

There are some special cases see [1]: "Unfortunately, traditional measures based on the Kullback–Leibler (KL) divergence and the Bhattacharyya distance do not satisfy all metric axioms necessary for many algorithms. In this paper we propose a modification for the KL divergence and the Bhattacharyya distance, for multivariate Gaussian densities, that transforms the two measures into distance metrics."

[1] K. Abou-Moustafa and F. Ferrie, "A Note on Metric Properties for Some Divergence Measures: The Gaussian Case," JMLR: Workshop and Conference Proceedings 25:1–15, 2012.

I think that answer to the question is possible. Because, recently in 2017 R. Farhadian showed that there is a probability on a heuristic sub set of integers that it is a metric. for his work, see the following link: http://journals.univ-danubius.ro/index.php/oeconomica/article/view/4010