2
$\begingroup$

In the E-step of the EM algorithm we maximize $$\max_\theta \sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z\mid\theta).$$ This expression is called the expectation of the complete data log-likelihood $\log p(X,Z\mid\theta)$. I do not see any expectation, which is defined as $E(Y)=\sum_YYp(Y)$. Why is it called this way? How can I see it is an expectation?

$\endgroup$
1
  • 1
    $\begingroup$ I'd write $\displaystyle \operatorname{E}(Y) = \sum_y y p(y),$ being careful about which $Y\text{s}$ are capital and which $y\text{s}$ are lower-case. $\endgroup$ Commented Apr 28, 2017 at 22:39

1 Answer 1

7
$\begingroup$

You are combining both steps. Breaking them out (e.g. see here), you have

E step

$Q(\theta\mid\theta_\text{old})=\sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z|\theta)$

M step

$\theta_\text{new}=\max_\theta Q(\theta\mid\theta_\text{old})$

For the "E step", you are computing the average $\mathbb{E}\big[\log p(X,Z\mid\theta)\big]$, taking $Z\sim p(Z\mid X,\theta_\text{old})$.

$\endgroup$
3
  • $\begingroup$ So this is the expectation $\mathbb{E}_Z$, i.e. with respect to $Z$ only? I appears that otherwise we would need to weight by $p(Z,X|\theta_{old})$. $\endgroup$
    – tomka
    Commented Apr 28, 2017 at 19:20
  • $\begingroup$ Yes, $X$ is the observed data which does not change. The average is over possible values of the hidden data $Z$. $\endgroup$
    – GeoMatt22
    Commented Apr 28, 2017 at 19:28
  • $\begingroup$ But that means that the "E-step" is not a step at all, in the usual meaning of this word, doesn't it? $\endgroup$ Commented Dec 14, 2018 at 14:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.