Derivation of gaussian mixture models assuming that hidden variable is known I saw the following notes from CS229 (screenshotted below). I am confused how the two equations are equivalent. How were they able to distribute the $log$ inside the summation? I don't see how knowing the hidden variable $z$, allows us to write it this way. 

 A: The meaning of $z^{(i)}$ in the two equations differs. In the observed log-likelihood
$$\ell^o(\theta|\mathbf x)=\sum_{i=1}^m \log \sum_{z^{(i)}=1}^k p(z^{(i)}|\theta)
p(x^{(i)}|z^{(i)},\theta)$$
$z^{(i)}$ is an index and the above could be written
$$\ell^o(\theta|\mathbf x)=\sum_{i=1}^m \log \sum_{j=1}^k p(j|\theta)
p(x^{(i)}|j,\theta)$$
In the completed log-likelihood
$$\ell^c(\theta|\mathbf x,\mathbf z)=\sum_{i=1}^m \log p(z^{(i)}|\theta)
p(x^{(i)}|z^{(i)},\theta)$$
$z^{(i)}$ is a latent variable that indicates the provenance component of $x^{(i)}$. 

The completed log-likelihood $\ell^c(\theta|\mathbf x,\mathbf z)$ is
  the log-likelihood associated with the joint distribution of the
  random sample $(\mathbf x,\mathbf z)$. The observed log-likelihood
  $\ell^c(\theta|\mathbf x)$ is the log-likelihood associated with the
  marginal distribution of the random sample $\mathbf x$.

A: The summation cannot be distributed but the product can. If the $z^{(i)}$ value is unknown, the variable inside the log is the summation over $z^{(i)}$. But when the value is known, it reduces to one term, which is just the product of the two probabilities.

To understand why it reduces to the one term, you need to think from a different point of view. The original log-likelihood is derived with the assumption that we don't know the index of the Gaussian in the mixture the observation comes from. This is an ignorant observer's point of view. The observer didn't see the drawing process of the sample and he/she only can guess probabilistically from what class the observation came.
However, there is a different point of view. If you are actually the person draw the sample, you perfectly know the origin of the observation. So in this case, you will have only one index of mixture contributing to the observation and the summation reduces to the one term. The EM algorithm goes with this view but it assumes we don't know the parameters of the distribution either and tries to decide the parameters in a recursive way.
