# Computing mixture of Binomial distributions

I'm trying to model a simple Bayes net, with $n$ samples based on a (unobservable) Bernoulli parameter, representing a true state of the world.

Let $T$ be a Bernoulli random variable, with probability $P(T=1) = y$.

Based on the realization of $T$, there is probability $p_t$: $p_0$ if $T=0$, $p_1$ if $T=1$. We then define $n$ Bernoulli random variables $X_1, \cdots, X_n$ , all with $P(X_i=1)=p_t$.

I'm interested in $Z=\sum_{i=2}^n X_i$, given an observation of $X_1$.

Q1: Is there a closed-form expression for $P(Z=z|X_1=x)$?

I'm also trying to understand how to compute the joint distribution on $\{X_i\}$ given the prior $P(X_i)$ and the pairwise conditional probability $P(X_j|X_i)$. Given the model, we know that the pairwise conditionals will be the same for all pairs $i,j$. However, it appears that I need further info to get the joint: $P(X_j=1|X_i=1)$ and $P(X_i=1)$ together determine $P(X_j=1|X_i=0)$ by Bayes rule, so that's only 2 free parameters, whereas the full model has 3 -- ($y, p_0, p_1$).

Q2: Given $P(X_i)$, and $P(X_j|X_i)$, what additional info about $\{X_i\}$ is needed to compute the joint distribution on $\{X_i\}$?

For convenience, let's change the notation to $P(X_i=1) = p$ where $p$ is a random variable taking on values $p_0$ or $p_1$. \begin{align} P(Z=z | p) &= \mathrm{Binomial}(n-1, p) \\ P(X_1=x | p) &= p^x (1-p)^{1-x} \\ P(p) &= \sum_t P(T=t) \delta(p - p_t) \\ P(X_1=x) &= \sum_t P(T=t) p_t^x (1-p_t)^{1-x} \\ P(Z=z | X_1=x) &= \frac{P(Z=z, X_1=x)}{P(X_1=x)} \\ &= \frac{\int_0^1 P(Z=z | p) P(X_1=x | p) P(p) dp}{P(X_1=x)} \\ &= \frac{\sum_t P(Z=z | p=p_t) P(X_1=x | p=p_t) P(T=t)}{P(X_1=x)} \end{align} The formula for $p(X_j | X_i)$ is the same thing but with $n=2$. The sum $X_1+X_2+X_3$ has 4 possible values, so its marginal distribution has enough information to recover the parameters of the model.
• Thanks! The first part makes sense, but I'm not quite following the last sentence--what do you mean by the marginal distribution of $X_1 + X_2 + X_3$? The marginals with respect to each $X_i$ separately? (I'm playing with the joint distribution on 3 $X$s, and it still looks like there's still a free parameter once I fix $P(X_i)$ and $P(X_j|X_i)$) – Victor Nov 12 '14 at 13:35
• Define a random variable $S = X_1 + X_2 + X_3$. I'm talking about the marginal distribution of $S$. – Tom Minka Nov 12 '14 at 19:05
• Ahh, ok. That makes sense. Sounds like you agree that just $P(X_i)$ and $P(X_j|X_i)$ isn't enough. I see how to recover the parameters from the marginal of $S$. Thanks for the help! – Victor Nov 12 '14 at 23:31