Derivation of EM in bishop

Question

I'm working through chapter 9 in Bishop (Mixture models and EM) and I'm stuck on equation 9.29. For those without the book:

Bishop states that the log likelihood for a latent variable model with

1. data, $X$
2. latent variables, $Z$
3. latent model parameters $\theta$

is given by:

$$ ln \; p(\textbf{X}| \boldsymbol{\theta}) = ln \; \left\{ \sum_{\textbf{Z}} p(\textbf{X,Z}|\boldsymbol{\theta} ) \right\} \: \: (1) $$

Shouldn't the log likelihood be over the data as well as the latent variables? Every example I've seen of a likelihood is over the data. If this is the case shouldn't the likelihood be:

$$ p(\textbf{X}| \boldsymbol{\theta}) = \prod_{ \textbf{x} \in \textbf{X}} \sum_{\textbf{Z}} p(\textbf{x,Z}|\boldsymbol{\theta} ) \: \: (2) $$

(multiplying over the data)

$$ ln \: p(\textbf{X}| \boldsymbol{\theta}) = \sum_{ \textbf{x} \in \textbf{X} } ln \left\{ \sum_{\textbf{Z}} p(\textbf{x,Z}|\boldsymbol{\theta} ) \right\} \: \: (3) $$

where $\textbf{x}$ is one observation in $\textbf{X}$

It strikes me that (1) is the equivalent of equation $9.7$ in Bishop (p430)

$$ p(\textbf{x}) = \sum_{k=1}^K \pi_k {N}(\textbf{x}| \boldsymbol{\mu} , \boldsymbol{\sigma}) \: \: (4)$$

Which is not described as a likelihood. When (4) is turned into a likelihood later on (equation 9.14), it is given as:

$$ ln\; p(X|\pi,\mu,\Sigma) = \sum_{n=1}^{N} ln \left\{ \sum_{k=1}^K \pi_k N(x_n|\mu_k,\Sigma_k) \right\} \: \: (5) $$

where Bishop sums over the latent variable $Z$ and the observations $x$.Which looks an awful lot like my attempt at the log likelihood (3)

My questions:

1. Surely if the progression from (4) to (5) holds shouldn't the log likelihood be (3) rather than (1) ?

2. Did bishop just miss out the sum over the data?

Answer 1

(1) is the general formula obtained using marginalisation. In (2), you assume iid data samples, so the joint distribution transforms into multiplication of marginals, e.g. $$p(\mathbf{X})=p(x_1,x_2)=p(x_1)p(x_2)$$ In the book, the sentence before equation 9.14, equation (5) in your question, states that the dataset is assumed to be iid.