2
$\begingroup$

I'm wondering how to find a posterior of a beta distribution when the "new information" is not an outcome of a binomial trial.

Let $p$ be the probability of Head of a (biased) coin toss. As usual in Bayesian inference, let $$p\sim Beta(a,b).$$

When the "new information" is Head or Tail, we can simply update $p$ by adding number of heads or tails to the shape parameters.

However, suppose that the new information I have is $$p\geq \frac{1}{2}.$$

If this is the case, how should I update the posterior in a Bayesian way?

In relation to the above question, and possibly more interestingly, for a Dirichlet distribution, if $$(p_1,p_2,p_3,p_4)\sim Dir(a,b,c,d)$$, what kind of Bayesian inference can be made out of the new information?: $$p_1+p_2\geq p_3+p_3$$

$\endgroup$
  • 1
    $\begingroup$ Not sure whether I have digested the question correctly. Why do not consider a uniform $(0.5,1)$ prior? $\endgroup$ – TPArrow Aug 29 at 10:23
4
$\begingroup$

As already noticed by @whuber in a comment to answer by @BruceET, this is not really a Bayesian scenario, since you don't seem to mention any data (nor any likelihood).

From what you are saying, you know that $p \sim \mathsf{Beta}(a, b)$, you also know that $p \ge 1/2$, what translates to knowing that $p$ is distributed according to beta distribution with parameters $a,b$ left truncated at $1/2$.

Same with the Dirichlet distribution, your knowledge that $p_1+p_2\geq p_3+p_3$ is a constraint about the distribution, not an "update" of the prior. Moreover, notice that this constraint leads to situation that may not be possible under Dirichlet distribution, so in fact the statements may be contradictory. The statement is in fact, that the $p_1, p_2, p_3, p_4$ are distributed according to distribution similar to Dirichlet, but constrained.

So...

  • If you are saying that for $p$ you assume truncated beta distribution as a prior, and want to use it together with some likelihood function and data, it is no more conjugate to binomial distribution, so you would need to use Markov Chain Monte Carlo for estimation. Defining truncated distribution can be done in any probabilistic programming framework, e.g. Stan, PyMC3, JAGS etc.
  • Same as above applies to the "Dirichlet"-like distribution, but since this is a custom distribution, it would be much more complicated (I have no easy solution for you).
  • If you are saying that the facts mentioned by you are the only information that you have and will have, and given this information you want to learn something about the distribution (e.g. expected value, quantiles), then this is a typical case of standard Monte Carlo simulation. For truncated beta, you could simply use inverse transform sampling, that is a simple and efficient way of sampling. For the "Dirichlet"-like distribution, it would again, be more complicated, but there are many possible approaches, starting from simple accept-reject sampling, ending at some more sophisticated solutions.
$\endgroup$
  • 1
    $\begingroup$ I see. Thank you for your comment. I guess I didn't clearly understand the differences between Bayesian inference and conditional distribution. So, the new information $p>\frac{1}{2}$ shouldn't be treated as "data" here. Instead, I should find a conditional distribution conditional on $p>\frac{1}{2}$. Same thing for the dirichlet case. I should find the distribution conditional on $p_1+p_2>p_3+p_4$. And if I want to make further bayesian inferences out of real data, I should consider using simulations as the distribution is no longer a conjugate. Do I understand it correctly? $\endgroup$ – Andeanlll Aug 30 at 1:45
  • $\begingroup$ @Andeanlll yes. $\endgroup$ – Tim Aug 30 at 4:34
2
$\begingroup$

This is not a standard way to obtain a posterior distribution in Bayesian inference (see Comment by @whuber). However, what about this, for the first part?

"Prior" is $p \sim \mathsf{Beta}(3, 4).$ "Data" is that $p > 0.$ "Posterior" is $\mathsf{Beta}(3, 4)$ truncated to $(1/2, 1).$

k = 1- pbeta(.5, 3, 4)
curve(dbeta(x,3,4)*(x>.5)/k, 0, 1, lwd=2, 
      n = 10001, ylab="Density", main="Posterior")
  abline(h=0, col="green2")

enter image description here

An alternative hint, in a more standard Bayesian setting, that $p > 1/2$ might have come from $n = 100$ binomial trials with $x = 70$ successes. In that case, the posterior would have been $\mathsf{Beta}(73, 34),$ plotted below.

enter image description here

$\endgroup$
  • 4
    $\begingroup$ By definition, the "data" are a realization of the random variable described by the prior. That's not the case here, so it's difficult to see why the information $p\gt 0.5$ can be viewed as data at all. $\endgroup$ – whuber Aug 29 at 18:51
  • 1
    $\begingroup$ Agreed. I probably should have done more than put data in quotes. $\endgroup$ – BruceET Aug 29 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.