I am trying to teach myself the expectation-maximization algorithm and the texts say the EM is particularly useful when the incomplete log-likelihood i.e. $P(X|\theta)$ where $\theta$ are the parameters of interest and $X$ is the observed data is difficult to optimize but the complete log-likelihood augmented with the hidden variables $Z$ is easier to optimize i.e. $\Sigma_{Z} P(X, Z|\theta)$ is easier to optimize.

I am having trouble figuring out in my head why the incomplete likelihood is more difficult to optimize. I cannot think of a case where somehow making the likelihood function more complicated by adding unobserved variables is making the problem simpler.

  • 2
    $\begingroup$ It doesnt say that the incomplete-data likelihood is more difficult to optimize! It says that when (in some concrete example) it is more difficult to optimize, and the completed-data likelihood is easy(ier) to maxmize, then you have a good case for using EM. I hope it also gives some example of this! $\endgroup$ – kjetil b halvorsen Jan 22 '15 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.