Confusion with the E-step of the EM algorithm for Gaussian Mixture Models

Question

So I was reviewing the E-step for the Gaussian Mixture model on Wikipedia.

And it looks like in the E-step all you really need to compute is the conditional distribution of Z because that is all that the M-step uses. However in the definition of the EM algorithm it states that in the E-step the Q function must be computed (i.e. the expected log likelihood)? Why doesn't it just say the conditional distribution of Z needs to be computed?

EDIT:

I just noticed that it also says in the article "Speaking of an expectation (E) step is a bit of a misnomer. What are calculated in the first step are the fixed, data-dependent parameters of the function Q. Once the parameters of Q are known, it is fully determined and is maximized in the second (M) step of an EM algorithm." So perhaps the answer is that it is just a misnomer i.e. it is not necessary to compute the full expected value.

Answer 1

Because they are not the same quantity. It is not the conditional distribution of $\mathbf{z}$, but rather it is an expectation taken with respect to this distribution. In other words, the quantity $Q(\theta \mid \theta_{\text{old}})$ is the expectation of $$ \log p(\mathbf{x}, \mathbf{z} \mid \theta) \tag{1} $$ taken with respect to $$ p(\mathbf{z} \mid \mathbf{x}, \theta_{\text{old}}), $$ where $\theta \neq \theta_{\text{old}}$.

The process is sort of intuitive. Usually you maximize (1), but you don't have access the hidden variables. So you pick some parameter $\theta_{\text{old}}$, then average out the hidden stuff. This gives you new function to maximize over all $\theta$: $Q(\theta \mid \theta_{\text{old}})$. Once you pick a new parameter, you keep just keep repeating the steps.

Answer 2

Since$$Q(\boldsymbol\theta\mid\boldsymbol\theta^{(t)}) = \Bbb{E}_{\mathbf{Z}\mid\mathbf{X},\boldsymbol\theta^{(t)}}\left[ \log L (\boldsymbol\theta; \mathbf{X},\mathbf{Z}) \right] = \int _\mathcal Z \log L (\boldsymbol\theta; \mathbf{X},\mathbf{z}) p(\mathbf{z}|\mathbf{X},\boldsymbol\theta^{(t)})\text{d}\mathbf{z}$$it is hard to see why, in general, the knowledge of the conditional distribution $ p(\mathbf{z}|\mathbf{X},\boldsymbol\theta)$ should be enough to find the next value $\boldsymbol\theta^{(t)}$ of $\boldsymbol\theta$ in the EM sequence.