# Does the Bhattacharyya distance and KL divergence measure the same thing?

I'm currently studying about the two methods in the title.
Does the Bhattacharyya distance and KL divergence measure the same thing, but in a different way?
I know the things that Bhattacharyya distance and KL divergence take into account when calculating is different, but in their essence, do they both measure difference between 2 probabilities distributions?

• KL divergence isn’t a distance.
– Sycorax
Mar 29, 2021 at 13:00
• @Sycorax but that's, as a concept (of course you can't compare them on equation), the same thing, right?
– Dave
Mar 29, 2021 at 13:26
• I suppose they measure the same thing in the same sense that swimming speed and bench press ability measure athleticism. // KL divergence is not symmetric and thus is not a metric, so calling it a distance (instead of a divergence) is iffy. (This is a different "Dave" posting.)
– Dave
Mar 29, 2021 at 13:31

Bhattacharyya distance and KL divergence both measure the dissimilarity between two probability distributions, but they do it in different ways. This means their numerical values are not directly comparable and, furthermore, they may not agree on the relative dissimilarities between multiple distributions. For example, suppose we have two distributions $$q_1, q_2$$ and want to determine which is closer to a reference distribution $$p$$. We might find that $$q_1$$ is closer to $$p$$ in terms of Bhattacharyya distance, but $$q_2$$ is closer in terms of KL divergence (or vice versa).
Bhattacharyya distance and KL divergence are both divergences, meaning they always take non-negative values, with a value of zero if and only if the two distributions are identical. However, neither is a true distance metric, which must additionally satisfy the triangle inequality (which neither measure does) and be symmetric. Bhattacharyya distance is symmetric ($$D_B(p,q) = D_B(q,p)$$), whereas KL divergence is not ($$D_{KL}(p \parallel q) \ne D_{KL}(q \parallel p)$$). The meaning of $$D_{KL}(p \parallel q)$$ is different than $$D_{KL}(q \parallel p)$$, and it matters a great deal which one we use.