2
$\begingroup$

If we are to choose between two hypotheses $H_0$ vs $H_1$, it seems natural to compare the posterior probabilities $P(H_0|D)$ vs $P(H_1|D)$ and pick the one with the larger probability. In other words, we can look at the posterior probability ratio $\frac{P(H_0|D)}{P(H_1|D)}$.

Yet, we often use the size of the Bayes factor to choose between hypotheses: $$ \frac{P(D|H_0)}{P(D|H_1)} = \frac{P(H_0|D)}{P(H_1|D)}/\frac{P(H_0)}{P(H_1)} $$

The two ratios are usually different because usually $P(H_0) \ne P(H_1)$ as $$ P(H_i) = \int_{\Theta_i} \pi(\theta) d\theta $$

I think choosing the hypotheses using the two criteria is equivalent to making a decision under $a_0-a_1$ loss with different $a_0, a_1$ values, but, still, I wonder why we would want to choose different $a0, a_1$ values for essentially the same task and why not choosing any other values.

I have seen people use the Bayes factor all the time, yet rarely see people use the posterior probability ratio (although it is mentioned in some textbooks). Why is that?

Improve this question
$\endgroup$
1

2 Answers 2

Reset to default
1
$\begingroup$

The Bayes factor formalises how the data change the prior ratio, so it measures what the data have to say about the hypotheses in a Bayesian framework, with the (hypothesis) priors "taken out" if you want. People can then use their own priors for making decisions or formalising their overall belief. One can also argue that the "evidence in the data alone" regarding the hypotheses is of interest in itself, even if ultimately posterior probabilities are used to make decisions.

A more problematic explanation is that many people using Bayesian statistics are very shy to specify informative priors, and convincing justifications of priors are hard to find. The Bayes factor lets them get away without doing so (although parameter priors are still needed for composite hypotheses).

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the great response. They all make sense. In addition, I am thinking of another justification. Since the Bayes factor can be written as $\frac{P(H_0|D)}{P(H_1|D)}/\frac{P(H_0)}{P(H_1)}$, it feels like I can interpret it as how much the posterior probability ratio "beats" the prior probability ratio. It is a bit like how much it can beat a baseline, where the prior ratio is the baseline. Let me know how you think of this. Thanks $\endgroup$
    – Tom Bennett
    Commented Oct 5, 2021 at 23:51
  • $\begingroup$ @TomBennett I don't quite get this, it sounds like a competition between them, but I don't see it as a competition. $\endgroup$
    – Christian Hennig
    Commented Oct 6, 2021 at 0:12
  • $\begingroup$ I may be wrong about this. I was thinking about what happens when the posterior probabilities and BF suggest conflicting things. For example, if the weight of animal ,$\theta$, has a prior that is uniformly distributed on $[0, 1000]$ and H0 is $[0, 1]$, H1 $(1, 1000]$. Then $P(H_0)/P(H_1)=\frac1{99}$. If posterior probability ratio is $P(H_0|D)/P(H_1|D)=\frac1{9}$, this gives a BF of 11 but $P(H_1|D)=0.9$. So if we look at BF, we are in favor of H0 but if we look at the posterior, we are in favor of H1. Maybe we should just report both so that people can apply their own priors if they want to $\endgroup$
    – Tom Bennett
    Commented Oct 6, 2021 at 13:39
  • 1
    $\begingroup$ Reporting both is surely not wrong. From my point of view it depends on the strength of your prior. If you have a prior that is well informed by all kinds of background knowledge that I don't have, I'll tend to trust your posterior. If the prior is just "uniform because I don't know any better" or any other default choice, your posterior doesn't really tell me more than your BF. (A heretic may then say, why do a Bayesian analysis in the first place?) $\endgroup$
    – Christian Hennig
    Commented Oct 6, 2021 at 14:32
  • $\begingroup$ Thanks a lot! It makes a lot of sense. This also means that as readers of these reports, we need to make a call on how much stock to put into the priors. $\endgroup$
    – Tom Bennett
    Commented Oct 6, 2021 at 14:51
2
$\begingroup$

I think $P(H_i)$ are just your prior belief of the hypotheses, not sure if that is what you try to depict here as I believe the integral you wrote should simply be equal to 1.

For example look at this R package which lets you specify prior odds for hypotheses/models .

I think most often practitioners assume these are equally likely and calculate Bayes factor as simply the posterior ratio .

Improve this answer
$\endgroup$
5
  • $\begingroup$ I think this is a bit different from model selection, in that the priors cannot be chosen, but instead are determined by the priors of the model. For example, let $\theta$ be the parameter that we are interested in and let it be the animal's weights. Let the two hypotheses be H0: $\theta<10$ pounds and H1: $\theta \ge10$. Then $P(H_1)=\int_{\theta<10} \pi(\theta) d\theta$, $P(H_2)=\int_{\theta \ge 10} \pi(\theta) d\theta$ and $P(H_1) + P(H_2)=1$. But we can't arbitrarily choose $P(H_1)$ and $P(H_2)$. By contrast, in model comparison, we often assume $P(M_0)=P(M_1)$. $\endgroup$
    – Tom Bennett
    Commented Oct 5, 2021 at 20:02
  • $\begingroup$ I've just haven't seen prior odds for hypotheses defined in this way. I guess it makes sense, but it breaks down for point hypotheses e.g. $H_0: \theta=0$ which seems to be a big issue $\endgroup$
    – bdeonovic
    Commented Oct 5, 2021 at 20:53
  • $\begingroup$ Yes, one has to do something special for point hypotheses. Typically people redefine the prior as $\pi(\theta) =\pi_0 \delta(\theta_0) + (1-\pi_0) \pi'(\theta)$. $\endgroup$
    – Tom Bennett
    Commented Oct 5, 2021 at 22:12
  • 1
    $\begingroup$ @TomBennett "we can't arbitrarily choose $P(H_0)$ and $P(H_1)$." Well, we can choose $\pi$ and the hypotheses probabilities by implication. We can also choose $\pi$ to make $P(H_0)=P(H_1)$, or even specify $\pi$ conditionally on $H_0, H_1$ with $P(H_0)$ and $P(H_1)$ chosen first. $\endgroup$
    – Christian Hennig
    Commented Oct 5, 2021 at 22:38
  • $\begingroup$ Good point. I should have said that the hypothesis priors are indirectly determined by the parameter priors $\pi(.)$. Yes, I've seen people doing a composite $\pi(.)$ with $P(H_i)$ chosen first, but don't know how to effectively use it and its full implications. $\endgroup$
    – Tom Bennett
    Commented Oct 5, 2021 at 23:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.