2
$\begingroup$

I have a random experiment with its probability space $(\Omega, \cal{F}, P)$ but no random variable defined over it. Is there any formal language for describing the variability of the random experiment? E.g. I feel the following three experiments have increasing variability, but I am not sure how to formalize this feeling (or if you have a different feeling, how to formalize that):

  1. $\Omega=\{\omega_1\}$, $\quad\ $ $Pr(\omega_1)=1$
  2. $\Omega=\{\omega_1,\omega_2\}$, $Pr(\omega_1)=0.9$, $Pr(\omega_2)=0.1$
  3. $\Omega=\{\omega_1,\omega_2\}$, $Pr(\omega_1)=Pr(\omega_2)=0.5$

I am not opposing to the idea of introducing a random variable if that helps. I am just saying there is none in the original setup.

$\endgroup$
5
  • 1
    $\begingroup$ Entropy comes immediately to mind, because the only thing that varies is the probability measure itself. Is that the sort of thing you mean by "formal language"? $\endgroup$
    – whuber
    Jun 9 '20 at 13:45
  • $\begingroup$ @whuber, yes, I think so. But when I checked Wikipedia's entry on entropy, I found entropy defined on a random variable, not the underlying experiment, hence my question. Can entropy be defined without reference to a random variable? $\endgroup$ Jun 9 '20 at 13:45
  • 1
    $\begingroup$ This is an unfortunate case where Wikipedia fails us. Entropy most fundamentally is a property of a discrete probability measure. Everything else is derived from that concept. $\endgroup$
    – whuber
    Jun 9 '20 at 13:48
  • $\begingroup$ @whuber, thank you! Occasions like this are one more reason for why I respect you so much and feel indebted! Are there any mainstream alternatives to entropy relevant to my question? $\endgroup$ Jun 9 '20 at 13:54
  • $\begingroup$ The concept of comparison of statistical experiments comes to mind. See the classical paper by Blackwell (of Rao-Blackwell fame). I don't have time for an answer now ... $\endgroup$ Sep 3 '20 at 13:09
1
$\begingroup$

A salient candidate is entropy. For a random experiment $X$ with possible outcomes $\omega_1,\dots,\omega_n$ and the corresponding probabilities $P(\omega_1),\dots,P(\omega_n)$, entropy is defined as $$ H(X) =-\sum_{i=1}^n P(\omega_i) \ln \left( P(\omega_i) \right). $$ It measures the average level of "information", "surprise", or "uncertainty" inherent in the experiment's possible outcomes.

The fact that Wikipedia's entry on entropy defines entropy w.r.t. a random variable rather than the underlying experiment can be considered a failure. According to @whuber,

This is an unfortunate case where Wikipedia fails us. Entropy most fundamentally is a property of a discrete probability measure. Everything else is derived from that concept.

$\endgroup$
2
  • $\begingroup$ Since entropy can be used as a measure of (bio)diversity, alternatively other measures of (bio)diversity could also be used. Obviously diversity is variation is variability! See for instance stats.stackexchange.com/questions/483535/… $\endgroup$ Jul 1 at 1:22
  • $\begingroup$ @kjetilbhalvorsen, thanks! I have seen and upvoted that answer already before. $\endgroup$ Jul 1 at 10:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.