So, I am trying Bayesian Linear Regression. Being new to this I have tried the following things:
Equation for generated data:
$$Y = \phi(X) \cdot W + \epsilon $$ where $$W \sim \mathcal{N} ( \overline{w} ,\Sigma_w)$$ $$\epsilon \sim \mathcal{N}(\overline{\epsilon}, \Sigma_\epsilon )$$
Joint probability of P(Y,W):
$$\begin{pmatrix}W \\ Y\end{pmatrix} \sim\mathcal{N}\left(\begin{pmatrix}\overline{w} \\\phi\overline{w} \end{pmatrix}, \begin{pmatrix} \Sigma_W & \Sigma_W\phi^T \\ \phi\Sigma_W & \phi\Sigma_W\phi^T + \Sigma_\epsilon\end{pmatrix}\right)$$
$\mu_{\text{post}}$ and $\Sigma_{\text{post}}$, the parameters for P(W|Y) are given by:
$$\mu_{\text{post}} = \overline{w}+ \Sigma_{\text{W}} \phi^T (\phi \Sigma_{\text{W}} \phi^T + \Sigma_\epsilon)^{-1} (Y - \phi \overline{w})$$ $$\Sigma_{\text{post}} = \Sigma_{\text{W}} - \Sigma_{\text{W}} \phi^T (\phi \Sigma_{\text{W}} \phi^T + \Sigma_\epsilon)^{-1} \phi \Sigma_{\text{W}}$$
What I observed was the W (mean of P(W|Y)) generated using this formulation is similar to the regularized closed form solution obtained through MAP by setting regularization parameter $\lambda= precision_{\text{prior}}/ precision_{\text{noise}}$.
This makes sense but makes me question why we need Bayesian linear regression for simpler cases.
