I came across the following in Kevin Murphy's "a probabilistic perspective on machine learning". I am struggling to understand the derivation of the conditional probability for $z_i$. I tried different forms of Bayes' formula try to derive this line (24.10), but failed so far. any hints much appreciated

enter image description here


$z_i$ is the indicator variable that corresponds to what Gaussan component the data point $\mathbf{x}_i$ belongs to.

(24.10) is the conditional probability of assigning data point $\mathbf{x_i}$ to component $k$, given the data point (of course), the components' weight vector $\boldsymbol{\pi}$, and the components' parameters $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$.

By applying Bayes rule, the conditional probability we are seeking is:

$$ p(z_i=k|\mathbf{x}_i,\boldsymbol{\pi},\boldsymbol{\mu},\boldsymbol{\Sigma}) = \frac{p(z_i=k|\pi_k)\;p(\mathbf{x}_i|z_i=k,\boldsymbol{\mu_k},\boldsymbol{\Sigma_k)}}{p(\mathbf{x}_i|\boldsymbol{\pi},\boldsymbol{\mu},\boldsymbol{\Sigma})}. $$

The denominator (as usual) does not depend on the value of $z_i$, so it's just a normalization factor. The first term of the numerator is just $\pi_k$ and the second term is the pdf of the Gaussian with parameters $(\boldsymbol{\mu}_k,\boldsymbol{\Sigma}_k)$ evaluated at $\mathbf{x}_i$, thus obtaining (24.10).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.