In the E-step of the EM algorithm we maximize $$\max_\theta \sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z\mid\theta).$$ This expression is called the expectation of the complete data log-likelihood $\log p(X,Z\mid\theta)$. I do not see any expectation, which is defined as $E(Y)=\sum_YYp(Y)$. Why is it called this way? How can I see it is an expectation?

  • 1
    $\begingroup$ I'd write $\displaystyle \operatorname{E}(Y) = \sum_y y p(y),$ being careful about which $Y\text{s}$ are capital and which $y\text{s}$ are lower-case. $\endgroup$ – Michael Hardy Apr 28 '17 at 22:39

You are combining both steps. Breaking them out (e.g. see here), you have

E step

$Q(\theta\mid\theta_\text{old})=\sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z|\theta)$

M step

$\theta_\text{new}=\max_\theta Q(\theta\mid\theta_\text{old})$

For the "E step", you are computing the average $\mathbb{E}\big[\log p(X,Z\mid\theta)\big]$, taking $Z\sim p(Z\mid X,\theta_\text{old})$.

| cite | improve this answer | |
  • $\begingroup$ So this is the expectation $\mathbb{E}_Z$, i.e. with respect to $Z$ only? I appears that otherwise we would need to weight by $p(Z,X|\theta_{old})$. $\endgroup$ – tomka Apr 28 '17 at 19:20
  • $\begingroup$ Yes, $X$ is the observed data which does not change. The average is over possible values of the hidden data $Z$. $\endgroup$ – GeoMatt22 Apr 28 '17 at 19:28
  • $\begingroup$ But that means that the "E-step" is not a step at all, in the usual meaning of this word, doesn't it? $\endgroup$ – Elmar Zander Dec 14 '18 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.