1
$\begingroup$

Consider for a unknown parameter $\Theta\in\mathbb{R}^n$ the following posterior density conditioned on a dataset $d=\{y_1, y_2\}$ of two generic measures $y_1,y_2 \in \mathbb{R}^p$ \begin{equation} p_{\Theta|D}(\theta) \triangleq p_{\Theta|Y_1, Y_2}(\theta| y_1,y_2) \qquad \forall \theta \in \mathbb{R}^n \end{equation} thanks to the Bayes theorem such posterior can be written as \begin{equation} p_{\Theta|D}(\theta)=\frac{\ell_D (\theta) \, p_\Theta (\theta)}{\int \ell_D (t) \, p_\Theta (t) \text{ d}t} \qquad \forall \theta\in\mathbb{R}^n \tag{1} \end{equation} where $p_\Theta (\theta)$ is a prior density and $\ell_D (\theta)$ is the joint likelihood of the two measurements $y_1,y_2$ \begin{equation} \ell_D (\theta) \triangleq p_{D|\Theta}(d|\theta) \triangleq p_{Y_1, Y_2 |\Theta}(y_1, y_2|\theta) \qquad \forall \theta\in\mathbb{R}^n \end{equation} suppose that the two measures are conditionally independent in order to simplify the likelihood as follows \begin{equation} \ell_D (\theta) = \underbrace{p_{Y_1|\Theta}(y_1|\theta)}_{\triangleq \ell_1(\theta)}\,\underbrace{p_{Y_2|\Theta}(y_1|\theta)}_{\triangleq \ell_2(\theta)} \qquad \forall \theta\in\mathbb{R}^n \end{equation} in this case the posterior $(1)$ can be written \begin{equation} p_{\Theta|D}(\theta)=\frac{\ell_1 (\theta) \, \ell_2 (\theta) \, p_\Theta (\theta)}{\int \ell_1 (t)\,\ell_2 (t) \, p_\Theta (t) \text{ d}t} \qquad \forall \theta\in\mathbb{R}^n \tag{2} \end{equation}

Now consider the following recursion:

  1. compute the "partial" posterior \begin{equation}p_{\Theta|Y_1}(\theta)\triangleq \frac{\ell_1 (\theta)\,p_\Theta(\theta)}{\int \ell_1 (t)\,p_\Theta(t) \text{ d}t} \qquad \forall \theta \in \mathbb{R}^n\end{equation}
  2. compute the "full" posterior \begin{equation}p'_{\Theta|D}(\theta)\triangleq \frac{\ell_2 (\theta)\,p_{\Theta|Y_1}(\theta)}{\int \ell_2 (t)\,p_{\Theta|Y_1}(t) \text{ d}t} \qquad \forall \theta \in \mathbb{R}^n \tag{3}\end{equation}

My question is, the "batch" posterior $p_{\Theta|D}$ expressed in $(2)$ is equivalent to the "recursive" posterior $p'_{\Theta|D}$ expressed in $(3)$?

I have te suspect that the answer is yes because, if we consider the non-normalized posteriors then we have for $(2)$: \begin{equation}P_{\Theta|D}(\theta)\triangleq \ell_2(\theta)\,\ell_1(\theta)\,p_\Theta(\theta) \qquad \forall \theta \in \mathbb{R}^n\end{equation} while for $(3)$: \begin{equation}P'_{\Theta|D}(\theta)\triangleq \ell_2(\theta)\,P_{\Theta|Y_1}(\theta)=\ell_2(\theta)\, \ell_1(\theta)\,p_\Theta(\theta) \qquad \forall \theta \in \mathbb{R}^n\end{equation} where $P_{\Theta|Y_1}(\theta)\triangleq \ell_1(\theta)\,p_\Theta(\theta)$. Clearly, $P'_{\Theta|D}=P_{\Theta|D}$. How can we deal with the normalizing factors?

$\endgroup$
2
  • $\begingroup$ Yes, this is a coherence of the Bayesian approach stressed in most textbooks. (The normalising constants vanish when need be.) $\endgroup$
    – Xi'an
    Oct 1 '20 at 16:47
  • 1
    $\begingroup$ See e.g. equation (1.4) in The Bayesian Choice. $\endgroup$
    – Xi'an
    Oct 1 '20 at 16:54
1
$\begingroup$

In (3), \begin{align}p'_{\Theta|D}(\theta) &= \frac{\ell_2 (\theta)\,p_{\Theta|Y_1}(\theta)}{\int \ell_2 (t)\,p_{\Theta|Y_1}(t) \text{ d}t}\\ &= \frac{\ell_2 (\theta)\,\frac{\ell_1 (\theta)\,p_{\Theta}(\theta)}{\int \ell_1 (\tau)\,p_{\Theta}(\tau) \text{ d}\tau}}{\int \ell_2 (t)\,\frac{\ell_1 (t)\,p_{\Theta}(t)}{\int \ell_1 (\tau)\,p_{\Theta}(\tau) \text{ d}\tau} \text{ d}t}\\ &=\frac{\ell_2 (\theta)\,\ell_1 (\theta)\,p_{\Theta}(\theta)}{\int \ell_2 (t)\,{\ell_1 (t)\,p_{\Theta}(t)} \text{ d}t}\\ &=p_{\Theta|D}(\theta)\end{align}

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.