4
$\begingroup$

I am trying to teach myself the expectation-maximization algorithm and the texts say the EM is particularly useful when the incomplete log-likelihood i.e. $P(X|\theta)$ where $\theta$ are the parameters of interest and $X$ is the observed data is difficult to optimize but the complete log-likelihood augmented with the hidden variables $Z$ is easier to optimize i.e. $\Sigma_{Z} P(X, Z|\theta)$ is easier to optimize.

I am having trouble figuring out in my head why the incomplete likelihood is more difficult to optimize. I cannot think of a case where somehow making the likelihood function more complicated by adding unobserved variables is making the problem simpler.

$\endgroup$
1
  • 2
    $\begingroup$ It doesnt say that the incomplete-data likelihood is more difficult to optimize! It says that when (in some concrete example) it is more difficult to optimize, and the completed-data likelihood is easy(ier) to maxmize, then you have a good case for using EM. I hope it also gives some example of this! $\endgroup$ Jan 22, 2015 at 15:25

1 Answer 1

1
$\begingroup$

It doesn't say that the incomplete-data likelihood is more difficult to optimize! It says that when (in some concrete example) it is more difficult to optimize, and the completed-data likelihood is easy(ier) to maxmize, then you have a good case for using EM. I hope it also gives some example of this!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.