I wish to optimize the following likelihood function for parameter $\Theta$:

$$p(D|\Theta)=\int_X\int_Y p(x, y, D|\Theta)dydx$$ where $X$ and $Y$ are latent variables and only $D$ is observed. I would like to use the Expectation-Maximization (EM) algorithm. If I understand it correctly, the E-step of the algorithm would be:

$$Q(\Theta|\Theta^{(t)})=\mathbb E_{X, Y|D,\Theta^{(t)}}[\log(p(X, Y, D|\Theta))]$$

However, I can only get samples from $p(X, Y|D,\Theta^{(t)})$ using Markov Chain Monte Carlo. So, I did the following:

$$Q(\Theta|\Theta^{(t)})=\mathbb E_{X, Y|D,\Theta^{(t)}}[\log(p(X, Y|D,\Theta))]+\log(p(D|\Theta))$$

Since $D$ is observed, I can calculate $\log(p(D|\Theta))$ from the data. Now here's my question: is it correct? And would you recommend any other way to do it or any other algorithm to use here?

  • $\begingroup$ Note: I corrected some $\Theta^{(t)}$ into $\Theta$ in the E-step. $\endgroup$
    – Xi'an
    Apr 19, 2020 at 10:52
  • $\begingroup$ Sounds like a case where you might apply variational inference $\endgroup$
    – jII
    Feb 16 at 1:32

1 Answer 1


The proposal is apparently negating the point in running an EM algorithm since, were $p(D|\theta)$ available in an analytic form, the completion by the latent variables would not be necessary since a numerical maximisation algorithm could be called upon.

Even within the EM algorithm framework the decomposition does not help since $$\log p(x,y|D,\theta) + \log p(D|\theta) = \log p(x,y,D|\theta)$$ does not modify the M-step.

When running a Monte Carlo EM (MCEM) version, the conditional expectation is replaced with $$\frac{1}{T} \sum_{\iota=1}^T \log p(x_\iota,y_\iota,D|\theta)\qquad (x_\iota,y_\iota)\sim p(x,y|D,\theta^{(t)})$$ where the decomposition is not useful either.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.