Updating a Bayes factor

Question

A Bayes factor is defined in Bayesian testing of hypothesis and Bayesian model selection by the ratio of two marginal likelihoods: given an iid sample $(x_1,\ldots,x_n)$ and respective sampling densities $f_1(x|\theta)$ and $f_2(x|\eta)$, with corresponding priors $\pi_1$ and $\pi_2$, the Bayes factor for comparing the two models is $$\mathfrak{B}_{12}(x_1,\ldots,x_n)\stackrel{\text{def}}{=}\frac{m_1(x_1,\ldots,x_n)}{m_2(x_1,\ldots,x_n)}\stackrel{\text{def}}{=}\frac{\int \prod_{i=1}^n f_1(x_i|\theta)\pi_1(\text{d}\theta)}{\int \prod_{i=1}^n f_2(x_i|\eta)\pi_2(\text{d}\eta)}$$ A book I am currently reviewing has the strange statement that the above Bayes factor $\mathfrak{B}_{12}(x_1,\ldots,x_n)$ is "formed by multiplying the individual ones [Bayes factors] together" (p.118). This is formally correct if one uses the decomposition \begin{align*}\mathfrak{B}_{12}(x_1,\ldots,x_n)&=\frac{m_1(x_1,\ldots,x_n)}{m_2(x_1,\ldots,x_n)}\\&=\frac{m_1(x_n|x_1,\ldots,x_{n-1})}{m_2(x_n|x_1,\ldots,x_{n-1})}\times \frac{m_1(x_{n-1}|x_{n-2},\ldots,x_1)}{m_2(x_{n-1}|x_{n-2},\ldots,x_1)}\times\cdots\\&\qquad\cdots\times\frac{m_1(x_1)}{m_2(x_1)}\end{align*} but I see not computational advantage in this decomposition as the update by$$\frac{m_1(x_n|x_1,\ldots,x_{n-1})}{m_2(x_n|x_1,\ldots,x_{n-1})}$$requires the same computational effort as the original computation of$$\frac{m_1(x_1,\ldots,x_n)}{m_2(x_1,\ldots,x_n)}$$outside artificial toy examples.

Question: Is there a generic and computationally efficient way of updating the Bayes factor from $\mathfrak{B}_{12}(x_1,\ldots,x_n)$ to $\mathfrak{B}_{12}(x_1,\ldots,x_{n+1})$ that does not require recomputing the entire marginals $m_1(x_1,\ldots,x_n)$ and $m_2(x_1,\ldots,x_n)$?

My intuition is that, besides particle filters, which indeed proceed along estimating the Bayes factors $\mathfrak{B}_{12}(x_1,\ldots,x_n)$ one new observation at a time, there is no natural way of answering this question.

Answer 1

Presumably the purpose of a recursive equation for Bayes factor would be when you have already calculated the Bayes factor for $n$ data points, and you want to be able to update this with one additional data point. It does seem that it is possible to do this without recomputing the marginals of the previous data vector, so long as the form of the posterior function $\pi_n$ is known. Assuming we know the form of this function (and assuming IID data as in your question), the predictive density can be written as:

$$\begin{equation} \begin{aligned} m(x_{n+1} | x_1,...,x_n) &= \int \limits_\Theta f(x_{n+1}|\theta) \pi_n(d \theta | x_1,...,x_n). \\[6pt] \end{aligned} \end{equation}$$

Hence, you have:

$$\begin{equation} \begin{aligned} m(x_1,...,x_{n+1}) &= m(x_1,...,x_n) \int \limits_\Theta f(x_{n+1}|\theta) \pi_n(d \theta | x_1,...,x_n). \\[6pt] \end{aligned} \end{equation}$$

Comparing two model classes via Bayes factor, we then get the recursive equation:

$$\begin{equation} \begin{aligned} \mathfrak{B}_{12}(x_1,...,x_{n+1}) &= \mathfrak{B}_{12}(x_1,...,x_{n}) \cdot \frac{\int _{\Theta_1} f(x_{n+1}|\theta) \pi_{1,n}(d \theta | x_1,...,x_n)}{\int _{\Theta_2} f(x_{n+1}|\theta) \pi_{2,n}(d \theta | x_1,...,x_n)}. \\[6pt] \end{aligned} \end{equation}$$

This still involves integration over the parameter range, so I agree with your view that there does not appear to be any computational advantage over just recomputing the Bayes factor via the initial formula you give. Nevertheless, you can see that this does not require you to recompute the marginals for the previous data vector. (Instead we compute the predictive densities of the new data point conditional on previous data, under each of the model classes.) Like you, I don't really see any computational advantage of this, unless it happens that this integral formula simplifies easily. In any case, I suppose it gives you another formula for updating the Bayes factor.

Answer 2

There is no computational advantage afaik but there is a conceptual one. By writing the Bayes factor as a product of partial Bayes factors and computing each one, we see how much each new observation should alter the odds, given the previous ones.