# Checking my understanding of Bayes factor hypothesis testing

I am new to Bayesian statistics. I am trying to understand my course notes on this topic. Here are the notes:

I will try to explain what these notes are saying and hopefully someone can correct my misunderstandings.

The opening two lines are relatively straightforward, because in Bayesian statistics we conceive of $$\theta$$ as a random variable, and we select whichever hypothesis has the higher probability, conditional on the data we've observed.

Regarding the Bayes factors, the Bayes factor against $$H_0$$ is

$$B = \frac{\frac{P(\theta \in \Theta_1 | y)}{P(\theta \in \Theta_0 | y)}}{\frac{P(\theta \in \Theta_1)}{P(\theta \in \Theta_0)}}.$$

(I found the $$B_{10(y)}$$ notation very confusing.) We choose $$H_1$$ if $$B > 1$$, or $$\log_{10} B > 0$$. Intuitively we are looking for the alternative hypothesis to gain on the null hypothesis when we update for the observed data.

Regarding the table of values given, I think I am supposed to reject $$H_0$$ for all these values. I found the table confusing because I guess my reflexes are more frequentist so when I read "evidence against $$H_0$$ is poor" I imagine something like a $$p$$-value of $$0.09$$, which is supposed to imply a level of evidence against $$H_0$$ that is non-zero but still too weak to reject $$H_0$$. However with this Bayes factor method, we more readily reject $$H_0$$ because we just require $$H_1$$ to be "better" (according to the definition of $$B$$) than $$H_0$$.

Am I making sense? I appreciate any help.

Rejecting hypotheses is mainly about making decisions under uncertainty. What counts as a reasonable threshold for rejection depends on what your goals are and your tolerance for potentially getting it wrong. A p-value of .09 might be too high to "reject" a null in some applications but acceptable in others, and you might very well consider a BF of 1.01 to be too close for your purposes.

With a BF of 1.01 (or p = .09), would you bet somebody lunch tomorrow against the null? Lunch for a whole week? Would you try to publish a paper based on that BF? What about strapping astronauts onto a million liters of liquid oxygen and blasting them into space? Etc etc.

So I think your instincts are roughly right here. BFs don't work quite the same way as p-values, but there's nothing magical about BFs such that nulls are automatically "rejectable" if there is any evidence against them at all. "How much" evidence often matters quite a lot.

There's another way to look at Bayes factors you might find interesting called the Savage-Dickey ratio. SDR is the ratio of the posterior mass to prior mass on the null hypothesis:

$$BF_{01} = \frac{p(\theta = \theta_{null} | D)}{p(\theta = \theta_{null})}$$

(Written as BF in favor of the null). If the posterior assigns less mass to the null than the prior, then the evidence disfavored the null.

 See Wagenmakers 2010 (PDF) Appendix A for detail.

• Thanks for your response. After reading this, I would say the main issue with what's written in my course notes is the oversimplification that "we choose $H_1$ if $B_{10(y)} > 1$". As you argue, it depends on the context, and I agree with this. The reference to the Savage-Dickey ratio seems to highlight to me (as a relative neophyte in statistics) that there is more than one way to think about what constitutes (strong) evidence. Thanks again. Dec 6 '20 at 6:30