I have multiple sets of discrete probability histograms(vectors) and I want to measure the distance between each histogram. I have done some research but I am in doubt.

Literature suggest I could use the Bhattacharyya distance or the Hellinger distance (both are closely related). Which one should I use.

Some additional info. prob Histogram(vector) $A= (.18, .61, .16, .05)$ and prob Histogram (vector) $B= (.26, .55, .16, .03)$. I want to calculate the distance/difference among these two probability Histograms. The Bhattacharyya coefficient ($BC$) =

$$ \sqrt{.18 \times .26}+\sqrt{.61 \times .55}+\sqrt{.16 \times .16}+\sqrt{.05 \times .03} \; . $$

Consequently the Bhattacharyya distance = $-\ln(BC)$ and Hellinger distance = $2 \sqrt{1-BC}$.

Is this the right measure and is this the right calculation?


Jensen-Shannon distance is the 1st thing I'd consider. If you don't insist on having a "distance function", you can directly use Jensen–Shannon divergence, from which this distance is derived.

JS divergence is widely used to measure the difference between two probability distributions. It fits your case, as the inputs are two probability vectors. JS divergence is a straightforward modification of the well-known Kullback–Leibler divergence.

Generally, KL and JS divergence require the input vectors have nonzero entries. In case of zeros in the input, many people simply choose to throw out those values. Check https://mathoverflow.net/a/72672 for more details on this issue.

| cite | improve this answer | |

There is also the Wasserstein metric, which has become more popular in the Bayesian inference literature. In your case, the 'wasserstein1d' function from package 'transport' should do the trick:

  x <- rnorm(200)
  y <- rnorm(150,2)
| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.