In the E-step of the EM algorithm we maximize $$\max_\theta \sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z\mid\theta).$$ This expression is called the expectation of the complete data log-likelihood $\log p(X,Z\mid\theta)$. I do not see any expectation, which is defined as $E(Y)=\sum_YYp(Y)$. Why is it called this way? How can I see it is an expectation?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ I'd write $\displaystyle \operatorname{E}(Y) = \sum_y y p(y),$ being careful about which $Y\text{s}$ are capital and which $y\text{s}$ are lower-case. $\endgroup$– Michael HardyApr 28, 2017 at 22:39
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
3
You are combining both steps. Breaking them out (e.g. see here), you have
E step
$Q(\theta\mid\theta_\text{old})=\sum_Z p(Z\mid X,\theta_\text{old})\log p(X,Z|\theta)$
M step
$\theta_\text{new}=\max_\theta Q(\theta\mid\theta_\text{old})$
For the "E step", you are computing the average $\mathbb{E}\big[\log p(X,Z\mid\theta)\big]$, taking $Z\sim p(Z\mid X,\theta_\text{old})$.
answered Apr 28, 2017 at 18:57
-
$\begingroup$ So this is the expectation $\mathbb{E}_Z$, i.e. with respect to $Z$ only? I appears that otherwise we would need to weight by $p(Z,X|\theta_{old})$. $\endgroup$– tomkaApr 28, 2017 at 19:20
-
$\begingroup$ Yes, $X$ is the observed data which does not change. The average is over possible values of the hidden data $Z$. $\endgroup$ Apr 28, 2017 at 19:28
-
$\begingroup$ But that means that the "E-step" is not a step at all, in the usual meaning of this word, doesn't it? $\endgroup$ Dec 14, 2018 at 14:10