# Comparing distribution A to B and C

I have three discrete probability distributions, A, B and C. They are all measuring P(X) under different circumstances. I suspect that A is more similar to B than it is to C. I know that I can compare the difference between distributions with KL divergence, but how can I test whether the difference between A-B is less than the difference between A-C?

• You have a tough problem, because statistical and mathematical theory will not decide the answer: you are assuming there is a relevant way to compare distributions so that "less than" has meaning. What meaning it might have is up to you to decide: that's not something we can tell you--although we can provide some guidance, if you would explain how you intend to interpret the result.
– whuber
Commented Jun 12, 2018 at 12:59
• You seem to answer your own question: by comparing the two KL-divergences. Of course that means you're committing to defining "difference between" as "KL-divergence from". Commented Jun 12, 2018 at 13:01
• Rather than comparing the distributions as a whole, can you not compare specific aspects of the disrubutions, captured by relevant quantiles or functions of quantiles? This might give you more insights into where the distributions differ (e.g., tails). Commented Jun 12, 2018 at 13:09