EM algorithm with dependent observations

Question

I am trying to implement an EM algorithm for dependent observations.

Specifically, I am dealing with families where the hidden variables $Z$ of the children are dependent on the hidden variables of the parents.

I have troubles calculating the expected value for $Z$, since for parents, it depends on the observed data for the child.

I'd like to ask if this has been done before, and if so, if someone could suggest some literature.

Alternatively, some hints on how I could tackle this would be very helpful too.

Edit: Since WIJ in the comments asked for the full model specification:

Let's say there is a response $X$ for each person that follows a gaussian distribution:

$$ X|Z=0 \sim N(4, 1) $$ $$ X|Z=1 \sim N(4+\alpha, 1) $$

A proportion of $p_1$ of the parents are in group 1. The group $Z=1$ is fully inheritable, that is, if at least one parent is $Z=1$, then the child is automatically $Z=1$.

The parameters $\theta$ for the EM algorithm are $\alpha$ and $p_1$.

I have trouble computing $\mathbb{E}(Z|X,\theta^{t})$ for children, since the observed data $X$ includes both parents. So for parents $i=1,2$ and a child $i=3$, I can not simply compute $\mathbb{E}(Z_3|X_3,\theta^{t})$, but have to use $\mathbb{E}(Z_3|X_1,X_2,X_3,\theta^{t})$, and I don't know how to calculate that.

Answer 1

I still do not quite understand your problem. But I will give an answer that I think will help anyway.

You seem to be saying that for each $Z^{(i)}$, there are multiple $(X^{(i)}_1, \ldots, X^{(i)}_m) := X^{(i)}$ associated with $Z^{(i)}$, and that you want to compute $\mathbb{E}[Z^{(i)}|X^{(i)}]$. Assume that $p(X^{(i)}|Z^{(i)}) = \prod_{j=1}^m p(X^{(i)}_j|Z^{(i)})$ i.e., $X^{(i)}_1, \ldots, X^{(i)}_m$ are i.i.d., conditioned on $Z^{(i)}$. Then,

$$\begin{eqnarray} p(Z^{(i)}|X^{(i)}) &=& p(X^{(i)}|Z^{(i)})p(Z^{(i)})/p(X^{(i)}) \\ &=& p(Z^{(i)}) \prod_{j=1}^m p(X^{(i)}_j|Z^{(i)})/p(X^{(i)}). \end{eqnarray}$$

All the terms on the right hand side are specified by your model. The normalizer is

$$p(X^{(i)}) = p(X^{(i)}|Z^{(i)}=0)p(Z^{(i)}=0) + p(X^{(i)}|Z^{(i)}=1)p(Z^{(i)}=1).$$