6
$\begingroup$

From "The Bayesian Choice" by Christian P. Robert.

The definition of the Bayes factor is given to be the ratio of the posterior probabilities of the null and the alternative hypothesis over the ratio of the prior probabilities of the null and alternative.

ie $$B_{01}^{\pi}= \frac{ \frac{P(\theta \in \Theta_{0}|x)}{P(\theta \in \Theta_{1}|x)}}{\frac{p(\theta \in \Theta_{0})}{ p(\theta \in \Theta_{1})}}$$

Which the author shortly simplifies to be

$$B_{01}^{\pi}=\frac{\int_{\Theta_{0}}f(x|\theta_{0})\pi_{0}(\theta)d\theta}{\int_{\Theta_{1}}f(x|\theta_{1})\pi_{1}(\theta)d\theta}$$

But when I try to write out all the terms and use that

$$\pi(\theta|x)=\frac{f(x|\theta) \pi(\theta)}{\int_{\Theta}f(x|\theta) \pi(\theta)d\theta}$$

I get

$$B_{01}^{\pi}=\frac{f(x|\theta_{0})\int_{\Theta_{1}}f(x|\theta_{1})\pi_{1}(\theta)d\theta}{f(x|\theta_{1})\int_{\Theta_{0}}f(x|\theta_{0})\pi_{0}(\theta)d\theta}$$

instead.

Anyone have any idea of where I went wrong? Probably some simple mistake I made or error.

$\endgroup$
5
$\begingroup$

My deepest apologies, there is a typo in the final expression! Indeed, here is the complete text from my book (p.231):

Definition 5.5. The Bayes factor is the ratio of the posterior probabilities of the null and the alternative hypotheses over the ratio of the prior probabilities of the null and the alternative hypotheses, i.e., $$ B^\pi_{01}(x) = {P(\theta \in \Theta_ 0\mid x) \over P(\theta \in \Theta_1\mid x)} \bigg/ {\pi(\theta \in \Theta_ 0) \over \pi(\theta \in \Theta_ 1)}. $$ This ratio evaluates the modification of the odds of $\Theta_0$ against $\Theta_1$ due to the \obs\ and can naturally be compared to $1$, although an exact comparison scale can only be based upon a loss function. In the particular case where $\Theta_0=\{\theta_0\}$ and $\Theta_1=\{\theta_1\}$, the Bayes factor simplifies to the usual likelihood ratio $$ B^\pi_{01} (x) = {f(x|\theta_0)\over f(x|\theta_1)}. $$ In general, the Bayes factor depends on prior information, but is still proposed as an ``objective'' Bayesian answer, since it partly eliminates the influence of the prior modeling and emphasizes the role of the observations. Actually, it can be perceived as a Bayesian likelihood ratio since, if $\pi_0$ is the prior distribution under $H_0$ and $\pi_1$ the prior distribution under $H_1$, $B^\pi_{01}(x)$ can be written as \begin{equation} B^\pi_{01} (x) = {\int_{\Theta_0} f(x|\theta_0)\pi_0(\theta) \,\text{d}\theta \over \int_{\Theta_1} f(x|\theta_1)\pi_1(\theta) \,\text{d}\theta} =\frac{m_0(x)}{m_1(x)}\,, \end{equation} thus replacing the likelihoods with the marginals under both hypotheses.

The prior is thus defined as a mixture: $$\pi(\theta)=\pi(\theta \in \Theta_ 0)\times\pi_0(\theta)\times\mathbb{I}_{\Theta_0}(\theta)+\pi(\theta \in \Theta_ 1)\times\pi_1(\theta)\times\mathbb{I}_{\Theta_1}(\theta)$$where $$\pi(\theta \in \Theta_ 0)=\rho_0\qquad\text{and}\qquad\pi(\theta \in \Theta_ 1)=1-\rho_0\stackrel{\text{def}}{=}\rho_1$$ are the prior weights of both hypotheses and $$\int_{\Theta_0} \pi_0(\theta_0)\text{d}\theta_0=\int_{\Theta_1} \pi_1(\theta_1)\text{d}\theta_1=1$$Therefore \begin{align*}P(\theta \in \Theta_ 0|x)=\int_{\Theta_0} \pi(\theta_0|x)\text{d}\theta_0&=\int_{\Theta_0} \pi(\theta_0|x)\text{d}\theta_0\\ &= \frac{\int_{\Theta_0} \pi(\theta_0)f(x|\theta_0)\text{d}\theta_0}{\int_{\Theta_0} \pi(\theta_0)f(x|\theta_0)\text{d}\theta_0+\int_{\Theta_1} \pi(\theta_1)f(x|\theta_1)\text{d}\theta_1}\\ &=\frac{\int_{\Theta_0} \rho_0\pi_0(\theta_0)f(x|\theta_0)\text{d}\theta_0}{\int_{\Theta_0} \rho_0\pi_0(\theta_0)f(x|\theta_0)\text{d}\theta_0+\int_{\Theta_1} \rho_1\pi_1(\theta_1)f(x|\theta_1)\text{d}\theta_1} \end{align*} and $$P(\theta \in \Theta_1\mid x)=\frac{\int_{\Theta_1} \rho_1\pi_1(\theta_1)f(x|\theta_1)\text{d}\theta_1}{\int_{\Theta_0} \rho_0\pi_0(\theta_0)f(x|\theta_0)\text{d}\theta_0+\int_{\Theta_1} \rho_1\pi_1(\theta_1)f(x|\theta_1)\text{d}\theta_1}$$Hence, $$\frac{P(\theta \in \Theta_ 0|x)}{P(\theta \in \Theta_1\mid x)}=\frac{\rho_0\int_{\Theta_0} \pi_0(\theta_0)f(x|\theta_0)\text{d}\theta_0}{\rho_1\int_{\Theta_1} \pi_1(\theta_1)f(x|\theta_1)\text{d}\theta_1}=\frac{\rho_0}{1-\rho_0}B^\pi_{01}(x)$$ This hopefully explains where the final expression comes from.

Alas, there is a mistake in the last ratio of integrals in the quoted text (presumably due to a cut & paste)

$$B_{01}^{\pi}=\frac{\int_{\Theta_{0}}f(x|\theta_{0})\pi_{0}(\theta)d\theta}{\int_{\Theta_{1}}f(x|\theta_{1})\pi_{1}(\theta)d\theta}$$

which should be (when identifying the integrands differently in the two integrals to signify that the integrals are over two different parameter spaces) $$B_{01}^{\pi}=\frac{\int_{\Theta_{0}}f(x|\theta_{0})\pi_{0}(\theta_0)d\theta_0}{\int_{\Theta_{1}}f(x|\theta_{1})\pi_{1}(\theta_1)d\theta_1}$$or (with another choice of representation of the integrands) $$B_{01}^{\pi}=\frac{\int_{\Theta_{0}}f(x|\theta)\pi_{0}(\theta)d\theta}{\int_{\Theta_{1}}f(x|\theta)\pi_{1}(\theta)d\theta}$$ The only case when the notations $\theta_0$ and $\theta_1$ are important is when both null and alternative hypotheses are point hypotheses, i.e., when $\Theta_0=\{\theta_0\}$ and $\Theta_1=\{\theta_1\}$, since both symbols then take specific values, like $\theta_0=3$ and $\theta_1=-2$. In this specific case, the posterior is concentrated on $\{\theta_0,\theta_1\}$ and the Bayes factor writes $$ B^\pi_{01} (x) = {f(x|\theta_0)\over f(x|\theta_1)}$$ since $$P(\theta=\theta_0|x)=\frac{\overbrace{\pi(\theta_0)}^{\rho_0}f(x|\theta_0)}{\pi(\theta_0)f(x|\theta_0)+\underbrace{\pi(\theta_1)}_{\rho_1}f(x|\theta_1)}$$

Thank you for pointing out this error, to be added to the list of typos.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.