Understanding the Jeffreys-Zellner-Siow (JZS) prior in Bayesian t-tests

Question

I am currently working on a lecture on Bayesian hypothesis testing, following the paper by Rouder et al, 2009. On Page 231 they present the formula for the relevant Bayes factor based on the Jeffreys-Zellner-Siow (JZS) prior in Equation 1. Here is the formula: $$B_{01} = \frac{ \left( 1 + \frac{t^2}{\nu}\right)^{-(\nu+1)/2} }{ \int_0^{\infty} (1 + N g)^{-1/2} \left(1+\frac{t^2}{(1 + N g) \nu} \right)^{-(\nu+1)/2} (2 \pi)^{-1/2} g^{-3/2} e^{-1/(2g)} \mathrm{d}g } $$

where $t$ is the usual t-statistic value, $N$ is the sample size, $\nu = N-1$ is the degrees of freedom, and $g$ is not defined in the paper as far as I can tell.

The authors also note that

To our knowledge, Equation 1 is novel. The derivation is straightforward and tedious and not particularly informative.

However, I would like to understand at least the basic intuition behind how they arrived at this formula to present the derivation in a simplified form to my students. I do understand the basic idea, namely that the Bayes factor $B_{01}$ is the ratio of the following marginal likelihoods:

$$M_0 = \int_0^{\infty} f_0(\mathbf{y} | \sigma^2) p_0(\sigma^2) \mathrm{d}\sigma^2$$

and

$$M_1 = \int_{-\infty}^{\infty}\int_0^{\infty} f_1(\mathbf{y} | \mu, \sigma^2) p_1(\mu, \sigma^2) \mathrm{d}\sigma^2 \mathrm{d}\mu$$

(see Page 229 of the paper).

My problem, probably due to my ignorance, is that it's unclear to me from the paper what the likelihoods $f_0$, $f_1$ and the prior distributions $p_0$, $p_1$ are in this particular case. Is the likelihood $f_0$ a Normal or a t-distribution density? Is $f_1$ a non-central t density? In the denominator ($M_1$), what is the prior $p(\mu, \sigma^2)$? And what is $g$ in the integral in the denominator of $B_{01}$?

For my purposes it would be completely sufficient if I could tell my students, "Look, $B_{01} = \frac{M_0}{M_1}$, and then these marginal likelihoods are obtained by integrating the product of the likelihoods $f_0$ and $f_1$ and the priors $p_0$ and $p_1$, respectively. The likelihoods are such-and-such distributions for this and that reason. The priors are such-and-such distributions for another set of reasons."

I don't need the exact steps leading to Equation 1, but I am hoping that if I plug in the correct likelihood and prior distributions in the marginal likelihood integrals $M_0$ and $M_1$ then at least I could reproduce Equation 1 with Mathematica.

Answer 1

As a start, $M_0,M_1$ specified in p. 229 are not any particular distributions but rather symbolize the different distributions under $H_0:\mu=0$ and $H_1:\mu\ne0$.

In the section relevant for equation 1, the discussion is whether the data comes from a centralized $t$ distribution (AKA Student's) or the noncentralized $t$ distribution. for the sake of this discussion, we formulate the hypotheses in a bit different manner: $H_0:\text{t is central},\quad H_1:\text{t is noncentral}$. The noncentrality here can be described using the location-scale transformation. Consider a noncentral $t$ variable with location $\mu$, scale $\sigma$ and df $\nu$. It is quite common to simplify the location-scale two variables setting into a single variable, $\delta=\mu/\sigma$. As Rouder et al. explain in p.231, they consider the case of unknown variance $\sigma^2$:

The intuition from the preceding discussion is that priors on parameters cannot be too variable, because they then include a number of unreasonable values that, when included in the average, lower the marginal likelihood. This intuition is critical for comparisons of models when a parameter enters into only one of the models, as is the case for parameter $\delta$. It is much less critical when the parameter in question enters into both models, as is the case for parameter $\sigma^2$.

Gönen et al. 2005 simplify the noncentral case to be $T_\nu(t|\mu,\sigma)=\frac{1}{\sigma}T_\nu\left(\frac{t}{\sigma}\middle| \frac{\mu}{\sigma},1\right)$. In addition, they give the Bayes Factor under assuming prior mean $\lambda=0$ (which is equivalent to assuming $\mu=0$ in our case):

$$BF=\left(\frac{1+t^2/\nu}{1+t^2/\nu(1+n_\delta\sigma^2_\delta)}\right)^{-\frac{\nu+1}{2}}\sqrt{1+n_\delta\sigma^2_\delta}$$

As they deal with the two-sample case and the factor $B_{01}$ in discussion refers to the one-sample case, we can write the above as

$$BF=\left(\frac{1+t^2/\nu}{1+t^2/\nu(1+N\sigma^2)}\right)^{-\frac{\nu+1}{2}}\sqrt{1+N\sigma^2}$$

This derivation is quite simple and is actually the ratio of centralized t pdf over location-scale transformed pdf.

Next, as Zellner & Siow 1980 recommend, the prior for $\sigma^2$ is $Inv-\chi^2(1)$, or equivalently $Inv-\Gamma(\frac{1}{2},\frac{1}{2})$. Finally, we mimic Liang et al. 2008 who use a $g$-prior (which is positive scalar) for the variance and use an $Inv-\Gamma(\frac{1}{2},\frac{1}{2})$ prior on $g$. So all that's left is to plug everything into the proper places.

For a specific value of $g$, the BF would be

$$BF(g)=\left(\frac{1+t^2/\nu}{1+t^2/\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}\sqrt{1+Ng}=\frac{\left(1+\frac{t^2}{\nu}\right)^{-\frac{\nu+1}{2}}}{(1+Ng)^{-\frac{1}{2}}\left(1+\frac{t^2}{\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}}$$

For our needs, we can write the conditional distribution $$p(t|g)\propto (1+Ng)^{-\frac{1}{2}}\left(1+\frac{t^2}{\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}$$ and so the joint distribution would be $p(t,g)\propto p(t|g)\pi(g)$. Our prior choice was $Inv-\Gamma(\frac{1}{2},\frac{1}{2})$ and so

$$\pi(g)=\frac{0.5^{0.5}}{\Gamma(0.5)}g^{-0.5-1}\exp\left(-\frac{0.5}{g}\right)=\frac{\sqrt{0.5}}{\sqrt{\pi}}g^{-1.5}\exp\left(-\frac{1}{2g}\right)=(2\pi)^{-\frac{1}{2}}g^{-\frac{3}{2}}e^{-\frac{1}{2g}}$$

The joint distribution is then

$$p(t,g)=(1+Ng)^{-\frac{1}{2}}\left(1+\frac{t^2}{\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}\cdot (2\pi)^{-\frac{1}{2}}g^{-\frac{3}{2}}e^{-\frac{1}{2g}}$$

In order to reflect $H_1$ we have to integrate out $g$ (which is positive) and so

$$\begin{aligned}M_1=p(t|H_1\text{ is true})=\int_{-\infty}^{\infty}p(t,g)dg\\=\int_0^{\infty}\left[ (1+Ng)^{-\frac{1}{2}}\left(1+\frac{t^2}{\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}\cdot (2\pi)^{-\frac{1}{2}}g^{-\frac{3}{2}}e^{-\frac{1}{2g}}\right] dg\end{aligned}$$

So finally the Bayes factor is exactly as in Equation (1) of Rouder et al.:

$$B_{01}=\frac{M_0}{M_1}=\frac{\left(1+\frac{t^2}{\nu}\right)^{-\frac{\nu+1}{2}}}{\int_0^{\infty}{\left[ (1+Ng)^{-\frac{1}{2}}\left(1+\frac{t^2}{\nu(1+Ng)}\right)^{-\frac{\nu+1}{2}}\cdot (2\pi)^{-\frac{1}{2}}g^{-\frac{3}{2}}e^{-\frac{1}{2g}}\right] dg}}.\blacksquare$$

---

Note: Considering the location-scale $t$ distribution with location $\mu$, scale $\sigma^2$ and df $\nu$, the pdf is

$$ p(x\ \mid\ \nu,\ \mu,\ \sigma^2) = \frac{\ \Gamma( \frac{\nu + 1}{2})\ }{\ \Gamma\left(\frac{\ \nu\ }{ 2 }\right)\ \sqrt{\pi\ \nu\ \sigma^2}\ }\ \left(\ 1 + \frac{\ 1\ }{ \nu }\ \frac{\ (x - \mu)^2\ }{\ \sigma^2\ }\ \right)^{-(\nu+1)/2}\ $$

so although it is a bit tedious, you can definitely construct all the equations taken from Gönen et al. 2005 by hand and obviously take $M_0=p(x\ \mid\ \nu,\ 0,\ 1)$ to represent the centralized $t$ distribution.