1
$\begingroup$

I am trying to implement an EM algorithm for dependent observations.

Specifically, I am dealing with families where the hidden variables $Z$ of the children are dependent on the hidden variables of the parents.

I have troubles calculating the expected value for $Z$, since for parents, it depends on the observed data for the child.

I'd like to ask if this has been done before, and if so, if someone could suggest some literature.

Alternatively, some hints on how I could tackle this would be very helpful too.

Edit: Since WIJ in the comments asked for the full model specification:

Let's say there is a response $X$ for each person that follows a gaussian distribution:

$$ X|Z=0 \sim N(4, 1) $$ $$ X|Z=1 \sim N(4+\alpha, 1) $$

A proportion of $p_1$ of the parents are in group 1. The group $Z=1$ is fully inheritable, that is, if at least one parent is $Z=1$, then the child is automatically $Z=1$.

The parameters $\theta$ for the EM algorithm are $\alpha$ and $p_1$.

I have trouble computing $\mathbb{E}(Z|X,\theta^{t})$ for children, since the observed data $X$ includes both parents. So for parents $i=1,2$ and a child $i=3$, I can not simply compute $\mathbb{E}(Z_3|X_3,\theta^{t})$, but have to use $\mathbb{E}(Z_3|X_1,X_2,X_3,\theta^{t})$, and I don't know how to calculate that.

Improve this question
$\endgroup$
6
  • $\begingroup$ What exactly is the full model specification? $\endgroup$ – wij Mar 4 '16 at 13:36
  • $\begingroup$ I updated the question - the model specification is at the bottom $\endgroup$ – Alexander Engelhardt Mar 4 '16 at 16:10
  • $\begingroup$ This looks like a mixture of 2 Gaussian distributions. Why is $\mathbb{E}(Z_3|X_1, X_2, X_3, \theta^t)$ not the same as $\mathbb{E}(Z_3| X_3, \theta^t)$? (Am I just confused?) $\endgroup$ – wij Mar 4 '16 at 17:48
  • $\begingroup$ Because the response for the child carries information on whether the parent belongs to group $Z=1$. If both parents have e.g. $X_1=5$ and $X_2=4$ but the kid has $X_3=12$, it's more probable that one of the parents come from group $Z=1$. $\endgroup$ – Alexander Engelhardt Mar 4 '16 at 18:11
  • $\begingroup$ Sorry but I am now completely lost on your terminology "parents", "children". Not sure which one is which. You have $p(X|Z, \alpha), p(Z|p_1)$. Do you have more? If for each $i$, $X_i$ has its own corresponding $Z_i \in \{0,1\}$ (i.e., $Z_i$ specifies which group 0 or 1 $X_i$ belongs), then you have a Gaussian mixture model. $\endgroup$ – wij Mar 4 '16 at 18:38
0
$\begingroup$

I still do not quite understand your problem. But I will give an answer that I think will help anyway.

You seem to be saying that for each $Z^{(i)}$, there are multiple $(X^{(i)}_1, \ldots, X^{(i)}_m) := X^{(i)}$ associated with $Z^{(i)}$, and that you want to compute $\mathbb{E}[Z^{(i)}|X^{(i)}]$. Assume that $p(X^{(i)}|Z^{(i)}) = \prod_{j=1}^m p(X^{(i)}_j|Z^{(i)})$ i.e., $X^{(i)}_1, \ldots, X^{(i)}_m$ are i.i.d., conditioned on $Z^{(i)}$. Then,

$$\begin{eqnarray} p(Z^{(i)}|X^{(i)}) &=& p(X^{(i)}|Z^{(i)})p(Z^{(i)})/p(X^{(i)}) \\ &=& p(Z^{(i)}) \prod_{j=1}^m p(X^{(i)}_j|Z^{(i)})/p(X^{(i)}). \end{eqnarray}$$

All the terms on the right hand side are specified by your model. The normalizer is

$$p(X^{(i)}) = p(X^{(i)}|Z^{(i)}=0)p(Z^{(i)}=0) + p(X^{(i)}|Z^{(i)}=1)p(Z^{(i)}=1).$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.