0
$\begingroup$

The following problem emerges from coordinate ascent variational inference in a mixture model with Dirichlet-Multinomial components. I want to compute the expectation of the log likelihood. Since my likelihoods are Dirichlet-Multinomials, that requires taking the expectation of a bunch of log(gamma function(random variable)) terms. How does one do this?

More specifically, let $\phi_k \sim Gamma(\alpha, \beta)$, where $\phi_k$ is the $k$th element of a $K$-dimensional vector $\phi$. How do I compute the following quantities?

  1. $$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \phi_k \right) \right] $$

$$\mathbb{E}_{\phi} \left[ \log \Gamma \left( \sum_{k=1}^K \phi_k \right) \right] $$

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ I don't think the first expectation has a closed form solution. Maybe Monte Carlo is a good strategy here? You can use the fact that the marginals are Beta. The second one looks easy enough though, since $\sum_{k=1}^K \phi_k=1$ is deterministic. $\endgroup$
    – PedroSebe
    Feb 13, 2022 at 22:42
  • $\begingroup$ Actually, your question just revealed to me that I was wrong to say $\phi \sim Dirichlet$. What is a reasonable distribution to place on $\phi$ if $\phi$ is the parameter for the log likelihood? $\endgroup$
    – Rylan Schaeffer
    Feb 13, 2022 at 22:51
  • $\begingroup$ Maybe $\phi_k$ should follow some Gamma distribution with hyperparameters $\alpha, \beta$? $\endgroup$
    – Rylan Schaeffer
    Feb 13, 2022 at 22:55
  • 1
    $\begingroup$ If ascent is what you are after, maybe use Jensen's inequality to lower bound both of these quantities and maximize the lower bound instead? $\endgroup$
    – mhdadk
    Feb 13, 2022 at 22:55
  • $\begingroup$ @mhdadk maximizing that lower bound is exactly where this expected value of the log likelihood arises from $\endgroup$
    – Rylan Schaeffer
    Feb 13, 2022 at 22:58

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.