In Zuk et al. 2012, they claim:

The set of probability distributions on $m$ values is the $m-1$-dimensional simplex denoted $S_m$.

Probability distribution functions are familiar to me. I understand the basic notion of a simplex as a geometric object, and I recall them from how the simplex method can be geometrically interpreted as a traversal of simplex nodes. But I am not familiar with representing sets of probability distributions as a simplex.

How does this representation work?


This comes from the definition of the standard simplex:

$$\left\{x \in \mathbb{R}^{k} : x_0 + \dots + x_{k-1} = 1, x_i \ge 0 \text{ for } i = 0, \dots, k-1 \right\} $$

It’s the set of $k$-dimensional points whose coordinates sum to 1 and are all non-negative. This lines up with the axioms of probability. The notion becomes clearer if you just replace each $x$ with a $p$.

As far as why it’s the $m-1$-simplex instead of $m$, consider the example of a distribution over three classes. The coordinates are points like $(0.3, 0.1, 0.6)$. These exist in a 3D space, but really they exist in a 2D subspace—a triangle whose vertices are at $(0,0,1)$, $(0,1,0)$, and $(1,0,0)$.

The vertices represent Kronecker distributions that place all mass on one of the three classes. Every distribution is a linear combination of these.


Not the answer you're looking for? Browse other questions tagged or ask your own question.