Simplifying equation of bayesian linear regression with gaussian priors

Question

I was reading the book Gaussian Processes for Machine Learning by C. E. Rasmussen & C. K. I. Williams. In the Regression chapter (chapter 2) they teach you how to do a linear regression for data $X \in \mathbb{R}^{n \times d}$ and $y \in\mathbb{R}^{n \times 1}$ in the sense of maximum likelihood with a gaussian prior on the weights and gaussian form assumed for data :

$p(y|w,X) \propto \exp(-\frac{1}{2\sigma_{n}^{2}}(y-Xw)^{T}(y-Xw))$

$p(w) \propto \exp(-\frac{1}{2}w^{T}\Sigma_{p}^{-1}w)$

So by direct application of Bayes rule you have :

$p(w|y,X) \propto p(y|w,X)p(w)$

$p(x|y,X)\propto \exp(-\frac{1}{2\sigma_{n}^{2}}y^{T}y+\frac{1}{2\sigma_{n}^{2}}y^{T}Xw+\frac{1}{2\sigma_{n}^{2}}w^{T}X^{T}y-\frac{1}{2\sigma_{n}^{2}}w^{T}X^{T}Xw-\frac{1}{2}w^{T}\Sigma_{p}^{-1}w)$

And then by some obscure magic you get:

$p(x|Y,X)\propto \exp(-\frac{1}{2}(w-\bar{w})^{T}(\frac{1}{\sigma_{n}^{2}}XX^{T}+\Sigma_{p}^{-1})(w-\bar{w}))$

with $\bar{w}=\sigma_{n}^{-2}(\sigma_{n}^{-2}XX^{T}+\Sigma_{p}^{-1})^{-1}Xy$.

I read somewhere it could come from the Woodbury formula indeed the final $\bar{w}$ is of the form $(A+UCV)^{-1}$ but I just cannot figure it out. There must be some simple trick to make sens out of it.

EDIT 1: I got to the result in the case where there is no $\Sigma_{p}$ but I am still at a loss regarding the general case I did something like:

$\forall W \in \mathbb{R}^{d \times 1}:$

$(y-Xw)^{T}(y-Xw)=(y-XW+XW-Xw)^{T}(y-XW+XW-Xw)=(y-XW)^{T}(y-XW)+(y-XW)^{T}(X)(W-w)+(W-w)^{T}X^{T}(y-XW)+(W-w)^{T}X^{T}X(W-w)$

As they are all scalar I can transpose the second term which gives:

$=||y-XW||_{2}^{2}+2(y-XW)^{T}X(W-w)+(w-W)^{T}X^{T}X(w-W)$

My equality still holds true for $W=\bar{w}$.

Then we do not care about the first term of the equality as it is constant w.r.t w, the second term evaluates to 0 if there is no $\Sigma_{p}$ but in the general case I end up with: $p(w|X,y) \propto \exp(-\frac{1}{2\sigma_{n}^{2}}\left(-2(y-X\bar{w})^{T}X(w-\bar{w})+(w-\bar{w})^{T}X^{T}X(w-\bar{w})\right)-\frac{1}{2}(w^{T}\Sigma_{p}^{-1}w))$
And I cannot manage to simplify it...

Answer 1

The answer I found is very complicated I do not know if there is a direct method (?). The expression above for the likelihood*prior can be "easily"(or at least in a more simple way) simplified using vector z=$(\beta, y)$:

$p(w|X,y)=\exp{z^{T}Rz}$

Then once you got there you use woodbury inversion formula to get the covariance of z. Then you use Theorem 4 part b to extract the covariance and mean of conditional $\beta|y$.

The process can be found in Bishop's Pattern and Recognition for ML Book on chapter 2 section 2.3.3..

I have no idea why my question was downvoted.