EM algorithm for Bivariate Normal Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's consider random variables $C_1,...,C_n$ which take the values $\{0,1,2\}$ and are independent of $X_i$. We observe each $C_i$ Now, if $C_i=0$, we observe $X_i$; if $C_i=1$, we observe only $U_i$; and finally if $C_i=2$, we observe $V_i$. I want to develop an EM algorithm to estimate the parameters of the distribution of $X_i$ given data.
My approach:
Let us define $Z_i = X_i$ if $C_i=0$ say, with probability ($\pi_1$), $Z_i=U_i$ if $C_i=1$ with probability $\pi_2$ and $Z_i=V_i$ if $C_i=2$ with probability $\pi_3$. Since, we observe both $Z_i,C_i$, the observed data likelihood is: $$L = \prod_{i=1}^{n} (\pi_1 f_{X}(x_i) + \pi_2 f_{U}(u_i) +\pi_3 f_{V}(v_i))$$.
In general, for EM algorithm we are required to find for the E-step: $E[\text{unobserved data}|\text{observed data}]$, but I am a bit confused with guessing what that should be in this case. In this case, if we observe $C_i=0$ then both $U_i,V_i$ are observed. However, if we observe $C_i=1$, then only $V_i$ is unobserved and for $C_i=2$, then $U_i$ is unobserved. So, does that mean our E step should deal with $E[V_i|U_i]$ and $E[U_i|V_i]$?
Can anyone help me out with this problem?
 A: Denoting the Normal parameters by $\theta$, the observed likelihood is
$$\pi_0^{n_0}\pi_1^{n_1}\pi_2^{n_2}\prod_{i:\,c_i=0}f_X(u_i,v_i;\theta)
\prod_{i:\,c_i=1}f_U(u_i;\theta)\prod_{i:\,c_i=2}f_V(v_i;\theta)$$
with each term being in closed (Normal) form. Hence this likelihood can be optimised in $(\theta,\pi)$ and does not require an EM algorithm.
If EM is implemented as a toy problem, the completed likelihood is
$$\pi_0^{n_0}\pi_1^{n_1}\pi_2^{n_2}\prod_{i:\,c_i=0}f_X(u_i,v_i;\theta)
\prod_{i:\,c_i=1}f_X(u_i,V_i;\theta)
\prod_{i:\,c_i=2}f_X(U_i,v_i;\theta)$$
where the missing / latent variables are upper cases. Removing the $\pi_j$'s in front which do not require at all EM to be optimised as
$$\hat\pi_i=n_i/n,$$
the E function (on $\theta$) is
$$\begin{align}\text{E}[\theta'|\theta,\mathcal D]&=
\sum_{i:\,c_i=0}\log f_X(u_i,v_i;\theta')+
\sum_{i:\,c_i=1}\mathbb E_\theta[\log f_X(u_i,V_i;\theta')|U_i=u_i]\\
&\qquad+
\sum_{i:\,c_i=2}\mathbb E_\theta[\log f_X(U_i,v_i;\theta')|V_i=v_i]\\
&=\sum_{i:\,c_i=0}\log f_X(u_i,v_i;\theta')\\
&\qquad+\sum_{i:\,c_i=1}\mathbb \log f_U(u_i;\theta')+
\sum_{i:\,c_i=1}\mathbb E_\theta[\log f_{V|U}(V_i;\theta',U_i=u_i])|U_i=u_i]\\
&\qquad+\sum_{i:\,c_i=2}\log f_V(v_i;\theta')+
\sum_{i:\,c_i=2}\mathbb E_\theta[\log f_{U|V}(U_i;\theta',V_i=v_i)|V_i=v_i]\end{align}$$
[where all density terms are normal] to be optimised in $\theta'$ (M-step).
Interestingly, albeit generically, this E-function also writes as
$$\overbrace{\sum_{i:\,c_i=0}\log f_X(u_i,v_i;\theta')+\sum_{i:\,c_i=1}\mathbb \log f_U(u_i;\theta')+\sum_{i:\,c_i=2}\log f_V(v_i;\theta')}^\text{observed log-likelihood}\\
\left.
\begin{align}&+\sum_{i:\,c_i=1}\mathbb E_\theta[\log f_{V|U}(V_i;\theta',U_i=u_i])|U_i=u_i]\\
&+\sum_{i:\,c_i=2}\mathbb E_\theta[\log f_{U|V}(U_i;\theta',V_i=v_i)|V_i=v_i]
\end{align}\right.$$
which means that close to (EM) convergence the expected part should not contribute significantly. When reaching a fixed point, i.e., when $\theta=\theta'$,
$$\mathbb E_\theta[\log f_{V|U}(V;\theta',U=u_i])|U=u_i]=-\frac{1}{2}$$
