# Formal language for describing variability of a random experiment

I have a random experiment with its probability space $$(\Omega, \cal{F}, P)$$ but no random variable defined over it. Is there any formal language for describing the variability of the random experiment? E.g. I feel the following three experiments have increasing variability, but I am not sure how to formalize this feeling (or if you have a different feeling, how to formalize that):

1. $$\Omega=\{\omega_1\}$$, $$\quad\$$ $$Pr(\omega_1)=1$$
2. $$\Omega=\{\omega_1,\omega_2\}$$, $$Pr(\omega_1)=0.9$$, $$Pr(\omega_2)=0.1$$
3. $$\Omega=\{\omega_1,\omega_2\}$$, $$Pr(\omega_1)=Pr(\omega_2)=0.5$$

I am not opposing to the idea of introducing a random variable if that helps. I am just saying there is none in the original setup.

• Entropy comes immediately to mind, because the only thing that varies is the probability measure itself. Is that the sort of thing you mean by "formal language"?
– whuber
Jun 9 '20 at 13:45
• @whuber, yes, I think so. But when I checked Wikipedia's entry on entropy, I found entropy defined on a random variable, not the underlying experiment, hence my question. Can entropy be defined without reference to a random variable? Jun 9 '20 at 13:45
• This is an unfortunate case where Wikipedia fails us. Entropy most fundamentally is a property of a discrete probability measure. Everything else is derived from that concept.
– whuber
Jun 9 '20 at 13:48
• @whuber, thank you! Occasions like this are one more reason for why I respect you so much and feel indebted! Are there any mainstream alternatives to entropy relevant to my question? Jun 9 '20 at 13:54
• The concept of comparison of statistical experiments comes to mind. See the classical paper by Blackwell (of Rao-Blackwell fame). I don't have time for an answer now ... Sep 3 '20 at 13:09

A salient candidate is entropy. For a random experiment $$X$$ with possible outcomes $$\omega_1,\dots,\omega_n$$ and the corresponding probabilities $$P(\omega_1),\dots,P(\omega_n)$$, entropy is defined as $$H(X) =-\sum_{i=1}^n P(\omega_i) \ln \left( P(\omega_i) \right).$$ It measures the average level of "information", "surprise", or "uncertainty" inherent in the experiment's possible outcomes.