Sequential updating of a Bayesian regression model I'd like to perform Bayesian regression of $Y$ on $X$. i.e. estimate a posterior $p\left(\theta\mid\mathcal{D}\right)$ of some regression parameter $\theta$ given a dataset $\mathcal{D}$ of pairs of points $(x_i, y_i)$. Suppose I have a dataset, $\mathcal{D}_1$, I estimate $p\left(\theta\mid\mathcal{D}_1\right)$, and then I get another dataset $\mathcal{D}_2$, and I want $p\left(\theta\mid\mathcal{D}_1, \mathcal{D}_2\right)$.
Intuitively, I want to use the posterior after the first update as the prior for the second. e.g. in linear regression with a Gaussian prior $\mu_{\theta},\Sigma_{\theta}$. I have a formula to which I insert $\mu_{\theta},\Sigma_{\theta}$ and $\mathcal{D}_1$ and it gives me $\mu_{\theta\mid\mathcal{D}_{1}},\Sigma_{\theta\mid\mathcal{D}_{1}}$. So when I subsequently input $\mu_{\theta\mid\mathcal{D}_{1}},\Sigma_{\theta\mid\mathcal{D}_{1}}$ and $\mathcal{D}_2$ it should give $\mu_{\theta\mid\mathcal{D}_{1}, \mathcal{D}_{2}},\Sigma_{\theta\mid\mathcal{D}_{1}, \mathcal{D}_{2}}$
I know how to justify this for sequential binary hypothesis testing, and I could probably show it for the specific example I gave above. But I know (maybe "feel" is a better word) that this is a general principle, just not sure how to prove it.
 A: Using Bayes' Theorem, we have:
\begin{equation}
p({\theta}|{y})=\dfrac{L({\theta}|{y})p({\theta})}{p({y})}.
\end{equation}
where $p({y})=\int{L({\theta}|{y})p({\theta})}d{\theta}$. Since $p({y})$ does not depend on ${\theta}$, the above expression can be rewritten as:
\begin{equation*}
p({\theta}|{y})\propto L({\theta}|{y})p({\theta}).
\end{equation*}
From this, we obtain:
$$p(\theta\mid D_1, D_2) = L(\theta\mid D_1, D_2)\times p(\theta)=L(\theta\mid D_1)\times L(\theta\mid D_2)\times p(\theta),$$
where $p(\theta)$ is a prior for $\theta$ before knowing $D_1$. This expression can be rewritten as:
$$L(\theta\mid D_2)\times (L(\theta\mid D_1)\times p(\theta))=L(\theta\mid D_2)\times p(\theta\mid D_1),$$
where $L(\theta\mid D_2)\times p(\theta\mid D_1)$ is actually the posterior of $\theta$ (after $D_2$) given prior $p(\theta\mid D_1)$.
A: Assuming that the data are conditionally independent given the parameter $\theta$,
$$p\left(\theta\mid\mathcal{D}_{1},\mathcal{D}_{2}\right) =p\left(\theta\right)\frac{p\left(\mathcal{D}_{1},\mathcal{D}_{2}\mid\theta\right)}{p\left(\mathcal{D}_{1},\mathcal{D}_{2}\right)}
 \\=p\left(\theta\right)\frac{p\left(\mathcal{D}_{1},\mathcal{D}_{2}\mid\theta\right)}{\int p\left(\mathcal{D}_{1},\mathcal{D}_{2}\mid\theta'\right)d\theta'}
 \\=p\left(\theta\right)\frac{p\left(\mathcal{D}_{1}\mid\theta\right)p\left(\mathcal{D}_{2}\mid\theta\right)}{\int p\left(\mathcal{D}_{1}\mid\theta'\right)p\left(\mathcal{D}_{2}\mid\theta'\right)d\theta'}
 \\=p\left(\theta\right)\frac{p\left(\mathcal{D}_{1}\mid\theta\right)}{\int p\left(\mathcal{D}_{1}\mid\theta'\right)d\theta'}\frac{p\left(\mathcal{D}_{2}\mid\theta\right)}{\int p\left(\mathcal{D}_{2}\mid\theta'\right)d\theta'}
 \\=p\left(\theta\right)\frac{p\left(\mathcal{D}_{1}\mid\theta\right)}{p\left(\mathcal{D}_{1}\right)}\frac{p\left(\mathcal{D}_{2}\mid\theta\right)}{p\left(\mathcal{D}_{2}\right)}$$
