# Batch posterior vs Recursive posterior

Consider for a unknown parameter $$\Theta\in\mathbb{R}^n$$ the following posterior density conditioned on a dataset $$d=\{y_1, y_2\}$$ of two generic measures $$y_1,y_2 \in \mathbb{R}^p$$ $$$$p_{\Theta|D}(\theta) \triangleq p_{\Theta|Y_1, Y_2}(\theta| y_1,y_2) \qquad \forall \theta \in \mathbb{R}^n$$$$ thanks to the Bayes theorem such posterior can be written as $$$$p_{\Theta|D}(\theta)=\frac{\ell_D (\theta) \, p_\Theta (\theta)}{\int \ell_D (t) \, p_\Theta (t) \text{ d}t} \qquad \forall \theta\in\mathbb{R}^n \tag{1}$$$$ where $$p_\Theta (\theta)$$ is a prior density and $$\ell_D (\theta)$$ is the joint likelihood of the two measurements $$y_1,y_2$$ $$$$\ell_D (\theta) \triangleq p_{D|\Theta}(d|\theta) \triangleq p_{Y_1, Y_2 |\Theta}(y_1, y_2|\theta) \qquad \forall \theta\in\mathbb{R}^n$$$$ suppose that the two measures are conditionally independent in order to simplify the likelihood as follows $$$$\ell_D (\theta) = \underbrace{p_{Y_1|\Theta}(y_1|\theta)}_{\triangleq \ell_1(\theta)}\,\underbrace{p_{Y_2|\Theta}(y_1|\theta)}_{\triangleq \ell_2(\theta)} \qquad \forall \theta\in\mathbb{R}^n$$$$ in this case the posterior $$(1)$$ can be written $$$$p_{\Theta|D}(\theta)=\frac{\ell_1 (\theta) \, \ell_2 (\theta) \, p_\Theta (\theta)}{\int \ell_1 (t)\,\ell_2 (t) \, p_\Theta (t) \text{ d}t} \qquad \forall \theta\in\mathbb{R}^n \tag{2}$$$$

Now consider the following recursion:

1. compute the "partial" posterior $$$$p_{\Theta|Y_1}(\theta)\triangleq \frac{\ell_1 (\theta)\,p_\Theta(\theta)}{\int \ell_1 (t)\,p_\Theta(t) \text{ d}t} \qquad \forall \theta \in \mathbb{R}^n$$$$
2. compute the "full" posterior $$$$p'_{\Theta|D}(\theta)\triangleq \frac{\ell_2 (\theta)\,p_{\Theta|Y_1}(\theta)}{\int \ell_2 (t)\,p_{\Theta|Y_1}(t) \text{ d}t} \qquad \forall \theta \in \mathbb{R}^n \tag{3}$$$$

My question is, the "batch" posterior $$p_{\Theta|D}$$ expressed in $$(2)$$ is equivalent to the "recursive" posterior $$p'_{\Theta|D}$$ expressed in $$(3)$$?

I have te suspect that the answer is yes because, if we consider the non-normalized posteriors then we have for $$(2)$$: $$$$P_{\Theta|D}(\theta)\triangleq \ell_2(\theta)\,\ell_1(\theta)\,p_\Theta(\theta) \qquad \forall \theta \in \mathbb{R}^n$$$$ while for $$(3)$$: $$$$P'_{\Theta|D}(\theta)\triangleq \ell_2(\theta)\,P_{\Theta|Y_1}(\theta)=\ell_2(\theta)\, \ell_1(\theta)\,p_\Theta(\theta) \qquad \forall \theta \in \mathbb{R}^n$$$$ where $$P_{\Theta|Y_1}(\theta)\triangleq \ell_1(\theta)\,p_\Theta(\theta)$$. Clearly, $$P'_{\Theta|D}=P_{\Theta|D}$$. How can we deal with the normalizing factors?

• Yes, this is a coherence of the Bayesian approach stressed in most textbooks. (The normalising constants vanish when need be.) Oct 1, 2020 at 16:47
• See e.g. equation (1.4) in The Bayesian Choice. Oct 1, 2020 at 16:54

In (3), \begin{align}p'_{\Theta|D}(\theta) &= \frac{\ell_2 (\theta)\,p_{\Theta|Y_1}(\theta)}{\int \ell_2 (t)\,p_{\Theta|Y_1}(t) \text{ d}t}\\ &= \frac{\ell_2 (\theta)\,\frac{\ell_1 (\theta)\,p_{\Theta}(\theta)}{\int \ell_1 (\tau)\,p_{\Theta}(\tau) \text{ d}\tau}}{\int \ell_2 (t)\,\frac{\ell_1 (t)\,p_{\Theta}(t)}{\int \ell_1 (\tau)\,p_{\Theta}(\tau) \text{ d}\tau} \text{ d}t}\\ &=\frac{\ell_2 (\theta)\,\ell_1 (\theta)\,p_{\Theta}(\theta)}{\int \ell_2 (t)\,{\ell_1 (t)\,p_{\Theta}(t)} \text{ d}t}\\ &=p_{\Theta|D}(\theta)\end{align}