I have multivariate normal distribution (MVN) and Bayesian linear model based on it. I understand what's going on here:
enter image description here
enter image description here
enter image description here

So I have a posterior x|y, where from prior knowledge x and observation y I estimate e. g. location of point from radar echo (posterior).
I used sequential update, i. e. using posterior after first estimation as a prior for the next one, and so on for all observations (with a simple for loop).

I have to get an equation for batch update and prove it, i. e. use all observations at once (as a vector of e. g. radar echos). I've only managed to get recursive formula, which I think is still sequential (and probably even more ineffective than a loop):

Sigma_0 = Sigma_x
u_0 = u_x

Sigma_(x|y)_n = [Sigma_(x|y)_(n-1) + (Sigma_y)^(-1)]^(-1)
u_(x|y)_n = Sigma_(x|y)_n * [(Sigma_y)^(-1) * y + Sigma_(x|y)_(n-1) * u_x]

Can you give me equation or some hints for the batch form? I know I'm missing some small detail here. I want to prove it through induction.


When adding one datapoint $y_1$, your posterior parameters are :

$\Sigma^{(1)}_{x|y} = [\Sigma_x + A^T\Sigma_y^{-1}A]^{-1}$ and $\mu^{(n)}_{x|y} = \Sigma^{(1)}_{y|x}[A^T\Sigma_y^{-1}(y_1-b)+\Sigma_x^{-1}\mu_x]$.

A nice property of Bayesian statistics is that adding a data point then updating the prior as the obtain posterior and adding another point and so forth for $n$ datapoints is the same as computing directly the posterior with $n$ datapoints from the prior.

So to get the posterior parameters from adding two points, you have to replace in the last expression $\Sigma_x$ by $\Sigma_{x|y}^{(1)}$ and $\mu_X$ by $\mu^{(1)}_{x|y}$. This gives:

$\Sigma^{(2)}_{y|y} = [\Sigma_x ^{-1} + 2 A^T\Sigma_y^{-1}A]^{-1}$ and $\mu^{(2)}_{x|y} = \Sigma^{(2)}_{x|y}[A^T\Sigma_y^{-1}(y_1 + y_2 - 2 b) + \Sigma_x^{-1}\mu_x]$

And you can easily show by induction that:

$\Sigma_{x|y}^{(n)} = [\Sigma_x ^{-1} + n A^T\Sigma_y^{-1}A]^{-1}$ and $\mu^2_{x|y} = \Sigma^{(n)}_{x|y}[n A^T\Sigma_y^{-1}(\overline{y} - b) + \Sigma_x^{-1}\mu_x]$

You can see that when $n\to\infty$ prior influence vanishes and posteriors parameters are equivalent to

$\Sigma_{x|y}^{(\infty)} \sim \frac{1}{n}A^{-1}\Sigma_y A^{-T}$ and $ \mu_{x | y}^{(\infty)} \to A^{-1} (\mathbb{E}({y}) - b)$.

I hope it answered your question.

  • $\begingroup$ Yes, thank you very much. $\endgroup$
    – qalis
    Mar 27 '20 at 20:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.