Find the correlation between two binary observations when the parameter is one of two possible values with priors $p(\theta_1)$ and $p(\theta_2)$ Consider two binary observations, $x_1$ and $x_2$, which are independent given $\theta$. Now suppose there is uncertainty with respect to $\theta$. Find the correlation between two binary observations when the parameter is one of two possible values with priors $p(\theta_1)$ and $p(\theta_2)$.
 A: By your assumption of conditional independence, $$
p(X_1=x_1,X_2=x_2\mid\theta=\theta_k) = p(X_1=x_1\mid\theta=\theta_k)\,p(X_2=x_2\mid\theta=\theta_k)
$$ 
where $x_i\in\{0,1\}$ and $\theta_k\in\{\theta_1,\theta_2\}$.
You can find the marginal distributions via
$$
p(X_1=x_1) = \sum_{k=1}^2 p(X_1=x_1, \theta=\theta_k)=\sum_{k=1}^2 p(X_1=x_1\mid \theta=\theta_k)p(\theta=\theta_k),
$$
and the same calculation to compute $p(X_2 = x_2)$. The joint distribution can be calculated as
$$
p(X_1=x_1,X_2=x_2) = \sum_{k=1}^2 p(X_1=x_1,X_2=x_2,\theta=\theta_k) = \sum_{k=1}^2 p(X_1=x_1,X_2=x_2\mid\theta=\theta_k)p(\theta=\theta_k).
$$
Now that you have all distributions of interest, you can calculate things like $\mu_{X_1}$ and $\mu_{X_2}$ (which will be equal). If you want to calculate the covariance,
$$
\text{Cov}(X_1,X_2) = E\left[(X_1-\mu_{X_1})(X_2-\mu_{X_2})\right] = E\left[X_1 X_2\right] - \mu_{X_1}\mu_{X_2}
$$
where
$$
E\left[X_1 X_2\right] = \sum_{x_1=0}^1 \sum_{x_2=0}^1 x_1 x_2 \, p(X_1=x_1,X_2=x_2).
$$
In this case,
$$
E\left[X_1 X_2\right] = p(X_1=1,X_2=1) = \sum_{k=1}^2 p(X_1=1,X_2=1\mid\theta=\theta_k)p(\theta=\theta_k)
=
\sum_{k=1}^2 p(X_1=1\mid\theta=\theta_k)p(X_2=1\mid\theta=\theta_k)p(\theta=\theta_k).
$$
Note that, in this case we have
$$
\mu_{X_1} = \sum_{x_1=0}^1 x_1 \, p(X_1=x_1) = p(X_1 = 1) = \sum_{k=1}^2 p(X_1 = 1\mid\theta=\theta_k)p(\theta=\theta_k).
$$
