3
$\begingroup$

Say we have two coins with unknown success probabilities $p_1$ and $p_2$. To know more about the probabilities, say that we use Bayesian approach.

To do so, we first set our prior: $P_1\sim Beta(1,1)$ and $P_2\sim Beta(1,1)$.

Tossing both of the coins together, we update each of the prior as usual.

For example, we ran 10 rounds of tossing, 4 Heads on coin 1 and 8 Heads on coin 2.

The posterior should be $P_1\sim Beta(5,7)$ and $P_2\sim Beta(9,3)$.

My question is that what happens if we receive an "extra" piece of information that says "$p_1$ is greater than $p_2$"

Is there any systematic way to accommodate this extra piece of information into the posterior?

$\endgroup$
2
  • 3
    $\begingroup$ You could simply constrain the support of the joint prior over $(P_1, P_2)$ to the set $\{ P_1 > P_2\}$. $\endgroup$
    – πr8
    Commented Aug 24, 2021 at 8:17
  • $\begingroup$ The joint density for $(p_1,p_2)$ would be proportional to $p_1^4(1-p_1)^6\,p_2^8(1-p_2)^2\,\mathbf I_{[0 \le p_2 < p_1 \le 1]}$ but the marginal densities would be more difficult to express $\endgroup$
    – Henry
    Commented Feb 16, 2023 at 11:09

2 Answers 2

2
$\begingroup$

What you're referring to in the first part of the question, is the beta-binomial model. where binomial distribution is assumed as the likelihood and beta as a prior, hence by conjugacy posterior is also a beta distribution.

Your problem description in the second part describes a different scenario because it is multivariate. If you know that $p_1 > p_2$, this means that the parameters are dependent and you are talking about some multivariate distribution for the parameters (vs two univariate beta distributions). In such a case, you cannot use two (independent) beta-binomial models. The constraint can be imposed by choosing a multivariate prior for the parameters. For such a model you won't have a closed-form solution, so you would need to use MCMC or some other kind of approximate inference.

$\endgroup$
3
  • $\begingroup$ Thank you for the answer. I got your point. So, the new information correlates the two separate inference problems and it makes the problem more complicated.. Then, alternatively, if we think about only one coin $p_1$ and if the new information is $p_1\geq 1/2$. If this is the case, what would be the resulting inference? $\endgroup$
    – Andeanlll
    Commented Aug 24, 2021 at 9:18
  • $\begingroup$ In our example, do we simply have $P_1\sim Beta(5,7)$ and $P_1\geq 1/2$ as the posterior? or we cannot have some thing like $P_1\sim Beta(5,7)$ as $Beta(5,7)$ has its mean lower than 1/2? $\endgroup$
    – Andeanlll
    Commented Aug 24, 2021 at 9:20
  • $\begingroup$ @Andeanlll I believe this thread answers your question stats.stackexchange.com/questions/538935/… $\endgroup$
    – Tim
    Commented Aug 24, 2021 at 9:31
2
$\begingroup$

For the case where you want to use the restriction $p_{1}, \geq 1/2$ your prior will be of the following form

$$\pi(p_{1})=\left\{\begin{matrix} Beta(a,b), & p_{1}\geq 1/2 \\ 0, & otherwise \end{matrix}\right. = Trunc-Beta(a,b)$$

Hence, your posterior can be derived as

$$\pi(p_{1}|x) \propto L(p;x)*Beta(p_{1};a,b)*\mathbb{I}(p_{1}\geq 1/2) = L(p;x)*Trunc-Beta(a,b)$$

The truncated Beta distribution can be calculated in the following way restricted to the interval $[1/2,1]$

$$Trunc-Beta(a,b) = \frac{Beta(a,b)}{F(1)-F(1/2)}$$ for exact calculations you can check here https://en.wikipedia.org/wiki/Beta_distribution.

Essentially what will happen to your posterior by adding this extra information $p_{1}\geq 1/2$, is that even though your likelihood might give importance to values of $p_{1}$ less than $1/2$ your prior will make them zero, because it gives zero mass to those values less than $1/2$.

In the case where you have $p_{1}\geq p_{2}$, then your prior will be of the form

$$\pi(p_{1},p_{2}) = \left\{\begin{matrix} f(p_{1},p_{2}), & p_{1}\geq p_{2} \\ 0,& otherwise \end{matrix}\right.$$

and the posterior

$$\pi(p_{1},p_{2}|x)\propto L_{1}(p_{1};x_{1})*L_{2}(p_{2};x_{2})*f(p_{1},p_{2})*\mathbb{I}(p_{1}>p_{2})$$ similarly, the product of the likelihoods $L_{1}(p_{1};x_{1})*L_{2}(p_{2};x_{2})$ might give importance to cases where $p_{1}<p_{2}$ but your prior will give again zero mass to those choices.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.