# Appropriate distance measure for comparing probability distributions

Suppose I want to compare some countries to see how similar they are with respect to the relative sizes of the industries in them. So I find data on the distribution of GDP across all industries for each country, and end up with a probability distribution over industries for each country. What distance measure should I use to compare these probability distributions?

 To clarify: I am looking for a distance measure that makes sense of our intuitive judgements about how similar countries are with respect to their industry breakdowns. For example, suppose that I have five countries (1, 2, 3, 4, 5) and four industries (A, B, C, D), and I want to compare Countries 2, 3, 4, and 5 with respect to how similar they are to Country 1.

Country % GDP from ind. A % GDP from ind. B % GDP from ind. C % GDP from ind. D
Country 1 40% 40% 20% 0%
Country 2 35% 45% 15% 5%
Country 3 0% 60% 30% 10%
Country 4 0% 80% 0% 20%
Country 5 0% 0% 80% 20%

The intuitive ranking is: Country 2 > Country 3 > Country 4 > Country 5.

• I think the difference between 10% and 20% for two countries with respect to some industry should probably count for the same as the difference between 80% and 90%. I can't think of a reason why it wouldn't.
• I assume a distance measure that uses ratios won't work very well given that there are zero-values?
• I don't want a measure that only takes into account the maximum difference between industries when you consider arbitrary industries from each country, since then Country 4 and Country 5 would end up counting as equally similar to Country 1.
• I don't want a measure that only takes into account the maximum difference when you compare two countries with respect to each industry, because then Country 3 and Country 4 would end up counting as equally similar to Country 1.
• But I think a measure that is based on the maximum difference when you compare two countries with respect to each combination of industries (as in Total Variation Distance?) might be ok?
• I think intuitively the measure should be symmetric; i.e., the degree to which Country x is similar to Country y = the degree to which Country x is similar to Country x.
• I guess one of the main things I don't understand is how I should weight differences between countries for the industries.
• Absolute difference in proportions - seems intuitive.
• Squared difference in proportions - what is the typical rationale for this? To give small differences even less weight relative to larger differences? I don't see why I would need to do this if the differences are already small to begin with.
• Squared difference in proportions divided by one of the proportions (like in Chi-squared distance) - what is the typical rationale for this? That the same sized difference is more significant when you're dealing with small values than when you're dealing with large values? I think maybe I don't care about that here?

Now I'm leaning towards Total Variation Distance or L1 norm. Does that sound like the right kind of approach?

I understand Euclidean distance, and using this for the comparisons seems intuitively plausible to me. However, Googling tells me that there are many other distance measures, and normally other ones are used to compare probability distributions. But, as someone without much background in statistics, I quickly get out of my depth when I try to understand whether and why they are suitable.

Could anyone point me in the right direction here? What do you think the most appropriate distance measure would be in this scenario, and why? (Also, if anyone can recommend any textbooks etc. that I can work through to get myself into a position where I can understand how to assess the merits of the myriad distance measures out there, and so answer a question like this for myself, please do.)

• Wasserstein, Kolmogorov-Smirnov, ... Commented Nov 10, 2023 at 12:31
• Can those measures be used to objectively quantify the direction of the difference? For example, one distribution is more shifted to the right than the other on the x axis? I heard about calculating the maximum vertical difference in the CDFs as a metric. Commented Nov 10, 2023 at 13:44
• The point is that there are myriad mathematical answers to your question. What the question lacks is essential information about why you are making this comparison: please explain to us what specific characteristic of "sizes of industries" you wish to compare and how you propose to quantify the degree of "similarity."
– whuber
Commented Nov 10, 2023 at 15:45
• @whuber I've edited the question to try to add some more detail, although I'm not sure I've fully understood what you're asking for (for example, I'm not sure what you mean by the 'characteristics' of the size of an industry). I'm measuring the relative sizes of the industries in terms of the proportion of GDP that they generate. As for how I propose to quantify the degree of similarity between countries, that's what my question is asking about, so I'm not sure how to add more detail there without already knowing the answer. Commented Nov 11, 2023 at 1:53
• You don't have an "x axis" along which a distribution can be "shifted." You have multinomial discrete probability distributions (one per country) over a defined set of industries, with no order among the industries. Please edit the question to say more about what's important to you for this measure between 2 countries: the maximum distance for any industry, or some overall measure? If the latter, how do you want to weight differences between countries for the industries (e.g., absolute or squared differences in proportions)? Do you want the measure to be symmetric between 2 countries?
– EdM
Commented Nov 11, 2023 at 21:19

To summarize the discussion in the comments:

Your "probability distribution" for each country can be considered as the probability that a unit of GDP originates in a specified industry. We'll assume that this is a well defined and meaningful value. As there is no inherent ordering among the industries, this might be considered a multinomial distribution within each country.

There is a very large set of statistically acceptable ways to construct a similarity (or distance) measure. The Wikipedia Statistical Distance page has links to over a dozen. This web page discusses 17. Some are true metrics in the technical sense. Other non-metric or even non-symmetric statistical distances can be useful in some applications; see this page, for example. Comments from @whuber point out the relationship between your situation and the more general problem of rank ordering, discussed in linked questions.

The choice among similarity/distance measures isn't really a statistical matter, however: it depends on what aspects of similarity are most important for you and for your field of application.

You are now thinking through what aspects of similarity matter the most to you. That's a very important start. When you have decided on those aspects, you should be able to find a similarity/distance measure that works for your application. You might try several, see which best conforms to your intuition, and then be prepared to justify your choice of measure when you report your results.

I suspect that others in your field have done similar studies and have already made suitable choices, choices that might be well established in your field and perhaps expected in publications. As a non-economist, I'd thus be reluctant to provide advice about your best choice. Consider asking for guidance on a site more devoted to economics, like Economics Stack Exchange.

If you are going to go beyond simple comparisons among countries, note that this is essentially an example of compositional data, in which all values among the categories for each case necessarily add up to 1 (or to 100%). This site has over 100 questions on that topic.