I came across the following in Kevin Murphy's "a probabilistic perspective on machine learning". I am struggling to understand the derivation of the conditional probability for $z_i$. I tried different forms of Bayes' formula try to derive this line (24.10), but failed so far. any hints much appreciated
1 Answer
$z_i$ is the indicator variable that corresponds to what Gaussan component the data point $\mathbf{x}_i$ belongs to.
(24.10) is the conditional probability of assigning data point $\mathbf{x_i}$ to component $k$, given the data point (of course), the components' weight vector $\boldsymbol{\pi}$, and the components' parameters $\boldsymbol{\mu}$ and $\boldsymbol{\Sigma}$.
By applying Bayes rule, the conditional probability we are seeking is:
$$ p(z_i=k|\mathbf{x}_i,\boldsymbol{\pi},\boldsymbol{\mu},\boldsymbol{\Sigma}) = \frac{p(z_i=k|\pi_k)\;p(\mathbf{x}_i|z_i=k,\boldsymbol{\mu_k},\boldsymbol{\Sigma_k)}}{p(\mathbf{x}_i|\boldsymbol{\pi},\boldsymbol{\mu},\boldsymbol{\Sigma})}. $$
The denominator (as usual) does not depend on the value of $z_i$, so it's just a normalization factor. The first term of the numerator is just $\pi_k$ and the second term is the pdf of the Gaussian with parameters $(\boldsymbol{\mu}_k,\boldsymbol{\Sigma}_k)$ evaluated at $\mathbf{x}_i$, thus obtaining (24.10).