8
$\begingroup$

I have been trying to find the expected value of a function of a random variable with a Dirichlet distribution by integrating its product with the Dirichlet density function over a simplex in R.

To check I was applying the correct function in R, I tried integrating the density function over the whole simplex, expecting to get 1, however I kept getting that the density function for a Dirichlet distribution with n categories integrated to sqrt(n) (using R package SimplicialCubature).

I assumed this must be wrong, but then I looked at the density function for 2 categories, consider the case where the alphas = (1,1). Then the density function is uniformly 1 (taking the density function from https://en.wikipedia.org/wiki/Dirichlet_distribution). So the integral of the density function over the 1-simplex just gives the length of the 1-simplex. But this is sqrt(2), as I found with the R code.

What am I missing here?

$\endgroup$
12
$\begingroup$

With two variables, you are defining a line segment in $\mathbb{R}^2$, as you pointed out. However, due to the simplex constraint, one of these two variables is redundant in terms of specifying the density, since there is a one-to-one relationship between $x_1$ and $x_2$. Therefore, the density is specified over $K-1$ free variables (i.e., in $\mathbb{R}$)

This is actually pointed out in the first line of this section of the Wikipedia article, albeit very subtly.

Therefore, your density function becomes:.

$$Dir_{1,1}(x_1,1-x_1)=\frac{\Gamma(2)}{\Gamma(1)^2}(x_1)^0(1-x_1)^0=1$$

Therefore,

$$\int_0^1 Dir_{1,1}(x_1,1-x_1) dx_1 = 1$$


Response to OP Comment

Due to the simplex constraints, the two-variable Dirichlet density is actually degenerate in $\mathbb{R}^2$, as shown by my construction above (it only requires one variable). While it is true it has a density of $1$, it does not have a density of $1$ on the line segment connecting $(1,0)$ with $(0,1)$. What the above construction shows is that the marginal density has a value of $1$. Your confusion comes from thinking of $x_2$ as a free variable, in which case the support of the Dirichlet on $\mathbb{R}^2$ would have a non-zero area. This intuition is fine in cases like the the bivariate gaussian, where the two variables are not perfectly correlated, but not in this case.

We can formally derive this as follows:

Let $L$ be some number in $[0,\sqrt{2}]$ specifying the distance from $(1,0)$ to $(0,1)$ along the connecting line segment. Thus, each value of $L$ identifies a unique $(x_ 1,x_2)$ pair. Using this notation, your assumption that the density is $1$ along this line boils down to:

$$P(L \in [a,b] \subset)=b-a$$

However, we can show this is not the case through a formal treatment of the joint density of $x_1,x_2$:

$$P_L(L\in [a,b])=P_{X_1,X_2}[(x_1,x_2) \in A_{[a,b]}]$$

Where $A_{[a,b]}:= \{(u,v): u \in [1-\frac{b}{\sqrt{2}},1-\frac{a}{\sqrt{2}}], v = 1- u]$

Now, let's calculate $P_L(L\in [a,b])$:

$$P_L(L\in [a,b])= \int_{A_{[a,b]}} dP_{X_1,X_2}= \int_{A_{[a,b]}} dP_{X_1}dP_{X_2|X_1} =\int_{A_{[a,b]}} 1 \;dP_{X_1} = \int_{1-\frac{b}{\sqrt{2}}}^{1-\frac{a}{\sqrt{2}}}1\; du = $$

$$\left(1-\frac{a}{\sqrt{2}}\right) - \left(1-\frac{b}{\sqrt{2}}\right) = \frac{1}{\sqrt{2}}(b-a)$$

Where the third equality comes about because $dP_{X_2|X_1} = 1$ for $X_2=1-X_1$ (i.e., its not a density, but a point probability mass at $1-X_1$)

As you can see, we've recovered the $\frac{1}{\sqrt{2}}$ normalizing constant for the density along the line segment in $\mathbb{R}^2$. Effectively, this (degenerate) joint density is just a linear transformation of one of the two marginals (either one will work). This results in the domain of the probability density to go from $1$ to $\sqrt{2}$, hence the density must decrease to compensate.

$\endgroup$
3
  • 1
    $\begingroup$ Many thanks, I agree with the logic of what you have written, but I can't square that in my mind with the fact that the function has constant value 1 and the line has length sqrt(2). So why shouldn't the integral give sqrt(2)?? $\endgroup$
    – EBartrum
    Oct 14 '15 at 9:38
  • $\begingroup$ @EBartrum I'll add some clarification around 7:30 EDT $\endgroup$
    – user75138
    Oct 14 '15 at 10:50
  • 3
    $\begingroup$ @EBartrum added some more details to round out the post (I know you accepted already, but others may want the extra details) $\endgroup$
    – user75138
    Oct 14 '15 at 13:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.