In its general form the E-step of the EM algorithm finds the expectation
$$ Q(\theta|\theta') =\int \log[ p(Y,Z | \theta)] p(Z|Y,\theta') d Z$$
where $Y$ the data, $Z$ the latent variables, $\theta'$ the current parameters, and $l(\theta|Y,Z) = p(Y,Z|\theta)$ the complete data likelihood.
My general question is: does EM require us to know the joint conditional (predictive) distribution of the latent variables, $p(Z|Y,\theta)$?
Context. This question is phrased with more context as follows. Suppose $Z=[X,S]$ has two random variables parameterised by distinct vectors $\lambda$ and $\phi$ respectively, i.e. we know (can evaluate and sample from) $p(X,S|\lambda,\phi)=p(X|\lambda)p(S|\phi)$. For my problem, data $Y$ are a deterministic function $Y=f(X,S)$.
Variables $X,S$ are marginally independent; however they are not independent conditional on $Y$. We also know the complete data likelihood, it is
$$p(Y,X,S|\lambda, \phi) = p(X,S|\lambda, \phi) = p(X|\lambda)p(S|\phi).$$
Also we know the full conditionals $p(X|S,Y,\lambda)$ and $p(S|X,Y,\phi)$. They denote truncated distributions which can be derived. If we fill this in we find:
$$ \int \log[ p(X|\lambda)p(S|\phi) ] p(X,S|Y, \phi',\lambda') dXdS$$
As noted by Xi'an the E step then considers
$$ Q(\lambda, \phi|\lambda',\phi') = E_{X,S|Y,\lambda',\phi'}[l(\lambda|X)] +E_{X,S|Y,\lambda',\phi'}[l(\phi|S)] $$
which could be maximized in turn in an expectation conditional maximization (ECM) algorithm. However, for the problem I am working on the joint predictive distribution $p(X,S|Y, \phi,\lambda)$ is not known. We can use the (known) truncated distributions to write
$$ p(X|S,Y, \lambda) p(S|Y,\phi) = p(S|X,Y, \phi) p(X|Y,\lambda)$$
However the marginals $p(X|Y,\lambda),p(S|Y,\phi)$ are then unknown for my problem. As Xi'an noted the log-likelihood simplifies to
$$ Q(\lambda, \phi|\lambda',\phi') = E_{X|Y,\lambda'}[l(\lambda|X)] +E_{S|Y,\phi'}[l(\phi|S)] $$
involving the same unknown densities.
Is there still a way to use a type of EM algorithm to solve this estimation problem?
Speculative procedure. Let us assume we know $S=s_0$ then we could use EM with
$$ \tilde{Q}_1(\lambda, \phi|\lambda',\phi') = E_{X|Y,s_0,\lambda'}[l(\lambda|X)] $$
Similarly if we knew $X=x_0$
$$ \tilde{Q}_2(\lambda, \phi|\lambda',\phi') = E_{S|Y,x_0,\phi'}[l(\phi|S)] $$
So I gauge that a procedure that updates as follows might be a valid EM?
- Sample $s_0 \sim p(S|\phi')$
- Update $\lambda'$ by maximizing $\tilde{Q}_1$
- Sample $x_0 \sim p(X|\lambda')$
- Update $\phi'$ by maximizing $\tilde{Q}_2$
Then iterate until convergence. This seems not pure EM as it uses data augmentation. But perhaps something like this is a known method?