4
$\begingroup$

I am currently going through the chapter 9 - Mixture Models and EM from Bishop's book - Pattern Recognition and Machine Learning (2006).

enter image description here

I could not understand the maximization step with respect to the mean-parameters. I am particularly referring to the equation (9.16).

$$0= -\sum_{1}^{n} \gamma(z_{nk} ) \Sigma_k (x_n - \mu_k)$$

Shouldn't that be $\Sigma_k^{-1}$, instead of $\Sigma_k $ ?

Also, a follow-up question in the same line of thought.

When we differentiate the likelihood with respect to $\Sigma$, how does he take care of the ${\lvert \Sigma\rvert} ^ {p/2}$ (where $p$ is the number of dimensions) that is present in the multivariate normal distribution? Looks like, from eq(9.19), he is simply ignoring the determinant term.

enter image description here

$\endgroup$
  • 6
    $\begingroup$ This error, and more, are corrected in the erratum $\endgroup$ – Nadiels May 20 '18 at 10:45
10
$\begingroup$

Indeed, indeed, there is a typo in (9.16) and it should be $$0=\sum_{n=1}^N \gamma(z_{nk}) {\mathbf \Sigma}_k^{-1} ({\mathbf x}_n-{\mathbf \mu}_k)$$Fortunately, this does not impact the next equation (9.17).

As for deriving the conditional MLE of the covariance matrix ${\mathbf \Sigma}_k$, the result and the method are correct. The determinant is accounted for when taking the derivative in ${\mathbf \Sigma}_k$. (If it was ignored, the lhs of (9.19) would be zero.) The steps are the same as in the Normal case, except for the weights $\gamma(z_{nk})$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.