1
$\begingroup$

I'm reading this chapter about EM (9.3.1) of the book "Pattern Recognition and Machine Learning".

I understand the basic EM algorithm for GMM, but I'm having some problems understanding the probabilistic interpretation of the E step.

Most of the following formulas make sense to me except 9.39. Can someone please explain it for me? Thanks a lot.

It seems that it is in the form of $$E[z_{nk}]=\frac{\sum_{z_{nk}}z_{nk}f(n,k)}{normalizer}$$ where the numerator is a summation over two possible states of $z_{nk}$ (0 and 1), so it becomes $\pi_kN(x_n|\mu_k,\Sigma_k)$ in the second step, am I right so far?

If so, what are we summing over here in the denominator? 0s and 1s of all possible $j$s (but the result doesn't quite match)? Why $f(n,k)=[\pi_kN(x_n|\mu_k,\Sigma_k)]^{z_{nk}}$ is used as an unnormalized probability?

It would make more sense if the denominator is the unnormalized zeroth moment and the numerator is the unnormalized first moment, did I understand it correctly?

enter image description here

enter image description here

$\endgroup$
1
$\begingroup$

The numerator is the summation over a $K$-dimensional one-hot vector, so the indicator variable only equal $z_{nk}$ to 1 once.

The denominator is the unnormalized zeroth moment, and can be shown to be equal to the summation over $K$ summations of such $K$-dimensional vectors.

See this question for more details Derivation of expectation-maximization in General - PRML

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.