Given a set of many of discrete probability distributions, is there a way I can efficiently calculate a metric that quantifies how different the entire set of these probability distributions are to each other?

Just to give an idea, assume there is 1 probability distribution that is distributed as $[0.7, 0.3, 0]$ over 3 discrete outcomes and another probability distribution in my set that is $[0.65, 0.25, 0.1]$, then on the basis of these two, I would want the metric to tell me that my set of distributions is not very variable. If there are three such similar probability distributions, the metric should tell me that the set is even less variable.

If I simply had 2 probability distributions, I could compute something like the KL divergence between them. But with 1000s of probability distributions, it would be extremely inefficient to compute the KL divergence between every pair. So I am wondering if there is an efficient way to take a matrix that represents the probability distributions, something like:

$$\begin{matrix}0.70, 0.30, 0.0 \\ 0.65, 0.25, 0.1 \\ ...\end{matrix}$$

And then give me a metric that captures the variability?

  • $\begingroup$ Could you explain where these distributions come from? How do you know them? Are the probabilities perhaps estimated from data? $\endgroup$ – whuber Nov 29 '18 at 14:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.