I'm trying to model a simple Bayes net, with $n$ samples based on a (unobservable) Bernoulli parameter, representing a true state of the world.

Let $T$ be a Bernoulli random variable, with probability $P(T=1) = y$.

Based on the realization of $T$, there is probability $p_t$: $p_0$ if $T=0$, $p_1$ if $T=1$. We then define $n$ Bernoulli random variables $X_1, \cdots, X_n$ , all with $P(X_i=1)=p_t$.

I'm interested in $Z=\sum_{i=2}^n X_i$, given an observation of $X_1$.

Q1: Is there a closed-form expression for $P(Z=z|X_1=x)$?

I'm also trying to understand how to compute the joint distribution on $\{X_i\}$ given the prior $P(X_i)$ and the pairwise conditional probability $P(X_j|X_i)$. Given the model, we know that the pairwise conditionals will be the same for all pairs $i,j$. However, it appears that I need further info to get the joint: $P(X_j=1|X_i=1)$ and $P(X_i=1)$ together determine $P(X_j=1|X_i=0)$ by Bayes rule, so that's only 2 free parameters, whereas the full model has 3 -- ($y, p_0, p_1$).

Q2: Given $P(X_i)$, and $P(X_j|X_i)$, what additional info about $\{X_i\}$ is needed to compute the joint distribution on $\{X_i\}$?


For convenience, let's change the notation to $P(X_i=1) = p$ where $p$ is a random variable taking on values $p_0$ or $p_1$. \begin{align} P(Z=z | p) &= \mathrm{Binomial}(n-1, p) \\ P(X_1=x | p) &= p^x (1-p)^{1-x} \\ P(p) &= \sum_t P(T=t) \delta(p - p_t) \\ P(X_1=x) &= \sum_t P(T=t) p_t^x (1-p_t)^{1-x} \\ P(Z=z | X_1=x) &= \frac{P(Z=z, X_1=x)}{P(X_1=x)} \\ &= \frac{\int_0^1 P(Z=z | p) P(X_1=x | p) P(p) dp}{P(X_1=x)} \\ &= \frac{\sum_t P(Z=z | p=p_t) P(X_1=x | p=p_t) P(T=t)}{P(X_1=x)} \end{align} The formula for $p(X_j | X_i)$ is the same thing but with $n=2$. The sum $X_1+X_2+X_3$ has 4 possible values, so its marginal distribution has enough information to recover the parameters of the model.

| cite | improve this answer | |
  • $\begingroup$ Thanks! The first part makes sense, but I'm not quite following the last sentence--what do you mean by the marginal distribution of $X_1 + X_2 + X_3$? The marginals with respect to each $X_i$ separately? (I'm playing with the joint distribution on 3 $X$s, and it still looks like there's still a free parameter once I fix $P(X_i)$ and $P(X_j|X_i)$) $\endgroup$ – Victor Nov 12 '14 at 13:35
  • $\begingroup$ Define a random variable $S = X_1 + X_2 + X_3$. I'm talking about the marginal distribution of $S$. $\endgroup$ – Tom Minka Nov 12 '14 at 19:05
  • $\begingroup$ Ahh, ok. That makes sense. Sounds like you agree that just $P(X_i)$ and $P(X_j|X_i)$ isn't enough. I see how to recover the parameters from the marginal of $S$. Thanks for the help! $\endgroup$ – Victor Nov 12 '14 at 23:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.