2
$\begingroup$

I am currently trying to figure out some strangeness about using the Binomial distribution in Bayesian modeling to define the likelihood.

To make an example assume I have two conditions, and in each condition I do five repeated measurements, each of which can be defined as a single Bernoulli trial. So let's just say I get the results $Y_{1,i}=(1,1,1,0,0)$ for the first condition and $Y_{2,i}=(1,0,0,0,0)$. I want to compare the hypotheses that both $Y_{1}$ and $Y_{2}$ have the same probability of producing a $1$ (H1) vs. that they have a different probability (H2). For simplicity, I assume equal prior probability of both hypotheses. Also, assume flat prior on all parameters.

So the first hypothesis can be parametrized by two probabilities $\theta_1$ and $\theta_2$. So for $Y_1$ using a Binomial distribution I get three $1$s out of five and therefore $P(N=3|\theta_1)={5 \choose 3}\theta_1^3(1-\theta_1)^2$. Similarly, for $Y_2$ I get one $1$ out of five and therefore $P(N=1|\theta_2)={5 \choose 1}\theta_2^1(1-\theta_2)^4$. Now to get the total probability of H1 independently of $\theta_1$ and $\theta_2$ I need to multiply the two and marginalize out the parameters (i.e. integrate over the prior). Since I can split the multidimensional integral, I can just integrate each probability separately and then integrate:

$\int_0^1 {5 \choose 3}\theta_1^3(1-\theta_1)^2\;d\theta_1= {5 \choose 3}B(4,3)={5 \choose 3}\frac{\Gamma(4)\Gamma(3)}{\Gamma(7)}= \frac{5!}{(3!)(2!)}\frac{(3!)(2!)}{6!}=1/6$ for $Y_1$

and

$\int_0^1 {5 \choose 1}\theta_2^1(1-\theta_1)^4\;d\theta_2= {5 \choose 1}B(2,5)={5 \choose 1}\frac{\Gamma(2)\Gamma(5)}{\Gamma(7)}= \frac{5!}{(1!)(4!)}\frac{(1!)(4!)}{6!}=1/6$ for $Y_2$

and therefore $P(H1)=1/36$.

For the second hypothesis I only need a single parameter $\theta_1$, and thus I get four $1$s out of 10 and therefore $P(N=4|\theta_1)={10\choose 4}\theta_1^4(1-\theta_1)^6$. Now again I marginalize out $\theta_1$ and thus I get

$P(H2)=\int_0^1{10\choose 4}\theta_1^4(1-\theta_1)^6\;d\theta_1= \frac{10!}{(4!)(6!)}\frac{(4!)(6!)}{11!}=1/11$

So hypothesis H2 seems more likely. But looking at the formulas, I find that I will get $P(H1)=1/36$ and $P(H2)=1/11$ independently of the observation, because all values determined by the numbers of $1$s completely cancel out.

If I instead use a Bernoulli likelihood I get (derivation only for H2)

$P(H2)=\int_0^1 \theta_1^4(1-\theta_1)^6=\frac{(4!)(6!)}{11!}$

and

$P(H1)=\frac{(3!)(2!)(1!)(4!)}{6!}$

Which is actually dependent on the observation and therefore seems more correct. Now I have seen people using Binomial distributions as the final step in the likelihood definition in Bayesian samplers. So the question is, when would this work, and when would it fail?

I can see, that this might work (however I am not sure) when one is trying to estimate the parameters of each of the two models for H1 and H2. However, I have also seen this in tutorials about Baysian model selection, where a discrete random variable is used to switch between the two models. As far as I understood this method, the discrete variable just compares the integrals for each of the models (i.e. the probabilities after marginalizing out the parameters). So in that case, I assume that I would just get results independent of the observation?

So when is summarizing the data and then using a Binomial distribution safe, and when will it fail?

$\endgroup$
2
$\begingroup$

In this computation, you are not comparing the same events:

  1. Under the Binomial model and $H_1$, $P(H_1)=1/36$ is actually $P(N_1=3,N_2=1|H_1)$
  2. Under the Binomial model and $H_2$, $P(H_2)=1/11$ is actually $P(N_1+N_2=4|H_2)$
  3. Under the Bernoulli model and $H_2$, $P(H_2)=\frac{(4!)(6!)}{11!}$ is actually $P(Y_1=(1,1,1,0,0),Y_2=(1,0,0,0,0,0)|H_2)$
  4. Under the Bernoulli model and $H_1$, $P(H_1)=\frac{(3!)(2!)(1!)(4!)}{6!}$ is actually $P(Y_1=(1,1,1,0,0),Y_2=(1,0,0,0,0,0)|H_1)$

While the last two probabilities are about the same event, this is not true for the first two probabilities. One should compare $P(N_1=3,N_2=1|H_1)$ and $P(N_1=3,N_2=1|H_2)$ to make them commensurable, in which case one recovers the same probabilities as in 3. and 4.

$\endgroup$
  • $\begingroup$ Ok, I figured something like that should be at play here, but I am not sure I understand your answer completely. The comparison of 1 vs 2 uses the assumption that $ P(H_1|N_1=3,N_2=1)= P(N_1=3,N_2=1|H_1)\cdot C$ and $P(H_2|N_1+N_2=4|H_2)=P(H_2|N_1+N_2 = 4)\cdot C$ for the same $C$. But it seems this assumption is incorrect? Because obviously $P(Y_1=(1,1,1,0,0),Y=(1,0,0,0,0)|H_1)\neq P(N_1=3,N_2=1|H_1)$. So one does not recover the same probabilities in (correct versions of) 1 & 2 as in 3 & 4, but only up to a multiplicative constant? $\endgroup$ – LiKao Dec 14 '18 at 10:09
  • $\begingroup$ Never mind, I think I have figured out the answer to my commet on my own. So $C$ in the first equation should be $C_1=P(H_1)/P(N_1=3,N_2=1)$ and $C_2=P(H_2)/P(N_1+N_2=4)$. Now if $P(H_1)=P(H_2)$ (posteriori), then obviously $C_1\neq C_2$, because $N_1+N_2=4$ covers the (atomic) event $Y_1=(1,1,1,1,0)$ and $Y_2=(0,0,0,0,0)$, which is not covered by $N_1=3$ and $N_2=1$. Correct? $\endgroup$ – LiKao Dec 14 '18 at 10:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.