# Batch Bayesian linear model learning with multivariate normal distribution

I have multivariate normal distribution (MVN) and Bayesian linear model based on it. I understand what's going on here:   So I have a posterior x|y, where from prior knowledge x and observation y I estimate e. g. location of point from radar echo (posterior).
I used sequential update, i. e. using posterior after first estimation as a prior for the next one, and so on for all observations (with a simple for loop).

I have to get an equation for batch update and prove it, i. e. use all observations at once (as a vector of e. g. radar echos). I've only managed to get recursive formula, which I think is still sequential (and probably even more ineffective than a loop):

Sigma_0 = Sigma_x
u_0 = u_x

Sigma_(x|y)_n = [Sigma_(x|y)_(n-1) + (Sigma_y)^(-1)]^(-1)
u_(x|y)_n = Sigma_(x|y)_n * [(Sigma_y)^(-1) * y + Sigma_(x|y)_(n-1) * u_x]


Can you give me equation or some hints for the batch form? I know I'm missing some small detail here. I want to prove it through induction.

When adding one datapoint $$y_1$$, your posterior parameters are :

$$\Sigma^{(1)}_{x|y} = [\Sigma_x + A^T\Sigma_y^{-1}A]^{-1}$$ and $$\mu^{(n)}_{x|y} = \Sigma^{(1)}_{y|x}[A^T\Sigma_y^{-1}(y_1-b)+\Sigma_x^{-1}\mu_x]$$.

A nice property of Bayesian statistics is that adding a data point then updating the prior as the obtain posterior and adding another point and so forth for $$n$$ datapoints is the same as computing directly the posterior with $$n$$ datapoints from the prior.

So to get the posterior parameters from adding two points, you have to replace in the last expression $$\Sigma_x$$ by $$\Sigma_{x|y}^{(1)}$$ and $$\mu_X$$ by $$\mu^{(1)}_{x|y}$$. This gives:

$$\Sigma^{(2)}_{y|y} = [\Sigma_x ^{-1} + 2 A^T\Sigma_y^{-1}A]^{-1}$$ and $$\mu^{(2)}_{x|y} = \Sigma^{(2)}_{x|y}[A^T\Sigma_y^{-1}(y_1 + y_2 - 2 b) + \Sigma_x^{-1}\mu_x]$$

And you can easily show by induction that:

$$\Sigma_{x|y}^{(n)} = [\Sigma_x ^{-1} + n A^T\Sigma_y^{-1}A]^{-1}$$ and $$\mu^2_{x|y} = \Sigma^{(n)}_{x|y}[n A^T\Sigma_y^{-1}(\overline{y} - b) + \Sigma_x^{-1}\mu_x]$$

You can see that when $$n\to\infty$$ prior influence vanishes and posteriors parameters are equivalent to

$$\Sigma_{x|y}^{(\infty)} \sim \frac{1}{n}A^{-1}\Sigma_y A^{-T}$$ and $$\mu_{x | y}^{(\infty)} \to A^{-1} (\mathbb{E}({y}) - b)$$.