Why doesn't PDF of Dirichlet Distribution seem to integrate to 1?

Question

I have been trying to find the expected value of a function of a random variable with a Dirichlet distribution by integrating its product with the Dirichlet density function over a simplex in R.

To check I was applying the correct function in R, I tried integrating the density function over the whole simplex, expecting to get 1, however I kept getting that the density function for a Dirichlet distribution with n categories integrated to sqrt(n) (using R package SimplicialCubature).

I assumed this must be wrong, but then I looked at the density function for 2 categories, consider the case where the alphas = (1,1). Then the density function is uniformly 1 (taking the density function from https://en.wikipedia.org/wiki/Dirichlet_distribution). So the integral of the density function over the 1-simplex just gives the length of the 1-simplex. But this is sqrt(2), as I found with the R code.

What am I missing here?

Answer 1

With two variables, you are defining a line segment in $\mathbb{R}^2$, as you pointed out. However, due to the simplex constraint, one of these two variables is redundant in terms of specifying the density, since there is a one-to-one relationship between $x_1$ and $x_2$. Therefore, the density is specified over $K-1$ free variables (i.e., in $\mathbb{R}$)

This is actually pointed out in the first line of this section of the Wikipedia article, albeit very subtly.

Therefore, your density function becomes:.

$$Dir_{1,1}(x_1,1-x_1)=\frac{\Gamma(2)}{\Gamma(1)^2}(x_1)^0(1-x_1)^0=1$$

Therefore,

$$\int_0^1 Dir_{1,1}(x_1,1-x_1) dx_1 = 1$$

---

Response to OP Comment

Due to the simplex constraints, the two-variable Dirichlet density is actually degenerate in $\mathbb{R}^2$, as shown by my construction above (it only requires one variable). While it is true it has a density of $1$, it does not have a density of $1$ on the line segment connecting $(1,0)$ with $(0,1)$. What the above construction shows is that the marginal density has a value of $1$. Your confusion comes from thinking of $x_2$ as a free variable, in which case the support of the Dirichlet on $\mathbb{R}^2$ would have a non-zero area. This intuition is fine in cases like the the bivariate gaussian, where the two variables are not perfectly correlated, but not in this case.

We can formally derive this as follows:

Let $L$ be some number in $[0,\sqrt{2}]$ specifying the distance from $(1,0)$ to $(0,1)$ along the connecting line segment. Thus, each value of $L$ identifies a unique $(x_ 1,x_2)$ pair. Using this notation, your assumption that the density is $1$ along this line boils down to:

$$P(L \in [a,b] \subset)=b-a$$

However, we can show this is not the case through a formal treatment of the joint density of $x_1,x_2$:

$$P_L(L\in [a,b])=P_{X_1,X_2}[(x_1,x_2) \in A_{[a,b]}]$$

Where $A_{[a,b]}:= \{(u,v): u \in [1-\frac{b}{\sqrt{2}},1-\frac{a}{\sqrt{2}}], v = 1- u]$

Now, let's calculate $P_L(L\in [a,b])$:

$$P_L(L\in [a,b])= \int_{A_{[a,b]}} dP_{X_1,X_2}= \int_{A_{[a,b]}} dP_{X_1}dP_{X_2|X_1} =\int_{A_{[a,b]}} 1 \;dP_{X_1} = \int_{1-\frac{b}{\sqrt{2}}}^{1-\frac{a}{\sqrt{2}}}1\; du = $$

$$\left(1-\frac{a}{\sqrt{2}}\right) - \left(1-\frac{b}{\sqrt{2}}\right) = \frac{1}{\sqrt{2}}(b-a)$$

Where the third equality comes about because $dP_{X_2|X_1} = 1$ for $X_2=1-X_1$ (i.e., its not a density, but a point probability mass at $1-X_1$)

As you can see, we've recovered the $\frac{1}{\sqrt{2}}$ normalizing constant for the density along the line segment in $\mathbb{R}^2$. Effectively, this (degenerate) joint density is just a linear transformation of one of the two marginals (either one will work). This results in the domain of the probability density to go from $1$ to $\sqrt{2}$, hence the density must decrease to compensate.

Answer 2

A clarification too long for a comment. Wikipedia says the density is "with respect to Lebesgue measure on the Euclidean space $\mathbb{R}^{K-1}$", which I find misleading, since I read that to mean, "identify the codimension-1 affine space $A \subseteq \mathbb{R}^K$ containing the simplex with $\mathbb{R}^{K-1}$, and use the resulting Lebesgue measure." This interpretation must be incorrect, as the OP notes. The actual measure to use is the push-forward measure of Lebesgue measure on $$A \subseteq \{(x_1, \dots, x_{K-1}) \in \mathbb{R}^{K-1}: \sum x_k \leq 1\}$$ under the map $$(x_1, \dots x_{K-1}) \mapsto \left(x_1, \dots x_{K-1}, 1 - \sum x_k\right).$$ The Jacobian of this map is indeed $\sqrt{K}$, which is essentially the content of the other answer.

Another way to say this is that the PDF is not defined with respect to Lebesgue measure on the simplex, as I am inclined to read Wikipedia to mean, but with respect to Lebesgue measure on the projection of the simplex into the space $\{x_K = 0\} \subseteq \mathbb{R}^K$. With this measure, the volume of the simplex is $\frac{1}{(K-1)!}$ and the PDF of the flat Dirichlet distribution $$f(x_1, \dots, x_K; 1) \equiv \Gamma(K) = (K-1)!$$ does indeed integrate to $1$.