EM algorithm maximize which objective function In Wikipedia EM algorithm section of Gaussian Mixture examples , there are two likelihood functions:


*

*incomplete-data likelihood function $L(\theta;\mathbf{x})$

*complete-data likelihood function $L(\theta; \mathbf{x},\mathbf{z})$
which do we optimize when performing EM algorithm, why? 
I was told by the tutor we optimize formula 2 as formula 1 won't give us the right clustering results. I don't fully agree with what he said. Sure we are optimize formula 2, I agree on that. But the reason is that we are not able to set derivative to zero directly for formula 1 according to Andrew Ng's notes (last paragraph on page 1). However, there's a way to get us out of the difficulty, using Jensen's inequality (page 3), that is, applying Jensen's inequality to formula 1. Calculations on formula 1 shows that formula 2 is indeed an intermediate step of formula 1. So formula 1 also can provide us the same results/right clustering as formula 2 as well. This is my understanding of the relations between formulas 1 and 2, please correct me if I've missed anything.
 A: Expectation-Maximization optimizes the incomplete-data log-likelihood, which is typically denoted by $L(\theta ; \mathbf{x})$. Which is slightly different w.r.t. what you stated in your first bullet.
Of course, since the logarithm is a monotonic increasing function, maximizing the log-likelihood is equivalent to maximizing the likelihood.
I just wanted to state this difference because the correct definition of $L(\theta ; \mathbf{x})$ and $L(\theta ; \mathbf{x}, \mathbf{z})$ plays an important role next. Thus, just to be clear:
$$
L(\theta ; \mathbf{x}) := \log p(\mathbf{x} ; \theta) \\
L(\theta ; \mathbf{x,z}) := \log p(\mathbf{x,z} ; \theta)
$$
As you mentioned, one cannot directly solve (not even locally solve) $\arg\max_\theta L(\theta ; \mathbf{x})$ in many cases (e.g. Gaussian Mixture Models). Instead, one can (locally) maximize a lower bound on $L(\theta ; \mathbf{x})$ in order to solve the optimization problem.
In the Expectation-Maximization algorithm, this lower bound is the expected value of the complete-data log-likelihood, 
$L(\theta ; \mathbf{x, z})$, with respect to $p(\mathbf{z} \mid \mathbf{x} ; \theta^{(t)})$. Notice in bold the difference w.r.t. what you wrote in your second bullet.
This lower bound is typically denoted as $Q(\theta ; \theta^{(t)})$:
$$
Q(\theta; \theta^{(t)}) = 
\mathbb{E}_{\mathbf{z}| \mathbf{x}; \theta^{(t)}}[ L(\theta; \mathbf{x}, \mathbf{z})] = 
\sum_{\mathbf{z}} p(\mathbf{z}| \mathbf{x}; \theta^{(t)}) L(\theta ; \mathbf{x}, \mathbf{z}) =
\sum_{\mathbf{z}} p(\mathbf{z}| \mathbf{x}; \theta^{(t)}) \log p(\mathbf{x}, \mathbf{z} ; \theta )
$$
EM directly maximizes this lower bound, which indirectly also maximizes $L(\theta ;  \mathbf{x})$.
You can find a proof that $Q(\theta ; \theta^{(t)})$ is a lower bound of $L(\theta ; \mathbf{x})$ in the Wikipedia or in Andrew Ng's document on Jensen's inequality that you referenced.
