The following problem emerges from coordinate ascent variational inference in a mixture model with Dirichlet-Multinomial components. I want to compute the expectation of the log likelihood. Since my likelihoods are Dirichlet-Multinomials, that requires taking the expectation of a bunch of log(gamma function(random variable)) terms. How does one do this?

More specifically, let $\phi_k \sim Gamma(\alpha, \beta)$, where $\phi_k$ is the $k$th element of a $K$-dimensional vector $\phi$. How do I compute the following quantities?

  1. $$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \phi_k \right) \right] $$

$$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \sum_{k=1}^K \phi_k \right) \right] $$

  • 1
    $\begingroup$ I don't think the first expectation has a closed form solution. Maybe Monte Carlo is a good strategy here? You can use the fact that the marginals are Beta. The second one looks easy enough though, since $\sum_{k=1}^K \phi_k=1$ is deterministic. $\endgroup$
    – PedroSebe
    Feb 13, 2022 at 22:42
  • $\begingroup$ Actually, your question just revealed to me that I was wrong to say $\phi \sim Dirichlet$. What is a reasonable distribution to place on $\phi$ if $\phi$ is the parameter for the log likelihood? $\endgroup$ Feb 13, 2022 at 22:51
  • $\begingroup$ Maybe $\phi_k$ should follow some Gamma distribution with hyperparameters $\alpha, \beta$? $\endgroup$ Feb 13, 2022 at 22:55
  • 1
    $\begingroup$ If ascent is what you are after, maybe use Jensen's inequality to lower bound both of these quantities and maximize the lower bound instead? $\endgroup$
    – mhdadk
    Feb 13, 2022 at 22:55
  • $\begingroup$ @mhdadk maximizing that lower bound is exactly where this expected value of the log likelihood arises from $\endgroup$ Feb 13, 2022 at 22:58


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.