Let $X_1,\dots,X_{k+1}$ be mutually independent random variables, each having a gamma distribution with parameters $\alpha_i,i=1,2,\dots,k+1$ show that $Y_i=\frac{X_i}{X_1+\cdots+X_{k+1}},i=1,\dots,k$, have a joint ditribution as $\text{Dirichlet}(\alpha_1,\alpha_2,\dots,\alpha_k;\alpha_{k+1})$

According to the above theorem, we can get Dirichlet random variables from Gamma distribution.
So, I think that it's possible to calculate log likelihood of Dirichlet Distribution using Gamma.

Let $\vec{X} = (X_1,X_2,...,X_n)$ be samples from $Gamma(\alpha_i, 1)$. Then we normalize $\vec{X}$ and get $\vec{Y}$.
Then, Let $A = log\,p(\vec{X}) = \sum_{i=1}^nlog\,p(X_i) = \sum_{i=1}^n [(\alpha_i-1)\,log\,X_i - X_i - log\,\Gamma(\alpha_i)]\,$
$B = log\,p(\vec{Y})=\Sigma_{i=1}^n[(\alpha_i-1)log\,Y_i-log\,\Gamma(\alpha_i)]+log\,\Gamma(\bar{\alpha}) = \Sigma_{i=1}^n[(\alpha_i-1)(logX_i-log\bar{X})-log\,\Gamma(\alpha_i)]-log\,\Gamma(\bar{\alpha})$.

With the calculation, I think the result says that $A \neq B$. I don't know why this happen. I think that dirichlet random variable $\vec{Y}$ comes from Gamma random variable $\vec{X}$. So, I think they have the same log likelihood, but they didn't. Would you expalin about this?


1 Answer 1


A Gamma sample of $X_i\sim\text{Ga}(\alpha_i,1)$ differs from a Dirichlet sample $(Y_1,\ldots,Y_n)\sim\text{Dir}(\alpha_1,\ldots,\alpha_n)$ and it is thus no surprise that the likelihood functions on $\mathbf \alpha$ differ.

More importantly, the Dirichlet sample contains less information about the $\alpha_i$'s than the Gamma sample since $\sum_i Y_i=1$. Intuitively, it looses one dimension. In other words, when observing the $X_i$'s, turning them into $Y_i$'s by a normalisation step is wasting away information. (The exact information matrix for both the Gamma and the Dirichlet involves digamma functions, hence making the comparison delicate.)

The quoted theorem is about simulation, exhibiting a connection between the distribution of the $X_i$'s and the distribution of the $Y_i$'s. But the $X_i$'s are latent variables in statistical terms, they are not observed.

  • $\begingroup$ Then why the above theorem states that Dirichlet random variables can be sampled from Gamma random variables? is there any possibility to get Dirichlet likelihood from Gamma random variables? $\endgroup$
    – alryosha
    Commented Sep 30, 2020 at 8:24
  • $\begingroup$ Simulation and statistical inference are two different things. It is possible to simulate Dirichlet variates using Gamma variates as a marginalisation step. But it is not possible to recover the original Gamma variates by looking at the Dirichlet variates since the transform is not a bijection. This explains why inference (and likelihood) differ in the two settings. $\endgroup$
    – Xi'an
    Commented Sep 30, 2020 at 8:32
  • $\begingroup$ Thank you very much for your detailed answer. Can you recommend any materials that elaborate the difference between simulation and inference, or any related topics? $\endgroup$
    – alryosha
    Commented Sep 30, 2020 at 8:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.