My question is:
Why the incomplete-data likelihood equal to formula 1?
Why it should not equal to formula 2?
I apologize for not being word-perfect in English.
I'm reading the book Pattern Recognition And Machine Learning
. And I was confused at the derivation of EM algorithm in General at Page 467.
If we denote all of the observed variables by X
and all of the hidden variables by Z
. Our goal is to maximize the likelihood function: $p(X\mid\theta)$.
Why our goal is equal to $$\sum_Z p(X,Z \mid \theta) \qquad (1)$$
Where,
$$\sum_z p(X,Z \mid \theta) = \sum_z \prod_n \sum_k z_n^k p(x_n\mid \mu_k, \Sigma_k)\pi_k$$
from cos 513: mixture models and em algorithm page 4. As Gaussian mixture model, $p(x_n \mid \theta)= \sum_k \pi_k p(x_n \mid \mu_k, \Sigma_k)$ means instance $x_n$ with probability $\pi_k$ generated by kth
Gaussian component.
And $z_n^k$ is a one of K variable, if instance $x_n$ was generated by kth
Gaussian component, $z_n^k = 1$; otherwise, $z_n^k=0$.
In my mind,
$$p(X\mid\theta) = \prod_n \sum_z \sum_k z_n^k p(x_n\mid \mu_k, \Sigma_k)\pi_k \qquad (2)$$
So, I think this was the marginal likelihood. Cause, likelihood should be the product of probabilities of observations.