# Mean of the posterior distribution in bayesian linear regression with infinitely broad prior

Currently reading from Christopher Bishop's Pattern Recognition and Machine Learning book about parameter distribution under a bayesian linear regression.

On page 153, the author deduces that the posterior distribution over weights for a model of the form $$y(\bf x, \bf w) + \varepsilon$$ after $$N$$ observations with $$y({\bf x}, {\bf w}) = {\bf w}^T{\bf \phi(x)}$$, $$p(\varepsilon) = \mathcal{N}(0, \beta)$$ and with a conjugate prior of the form

$$p({\bf w}) = \mathcal{N}({\bf w} | {\bf m}_0, {\bf S}_0)$$

is

\begin{align} {\bf m}_N &= {\bf S}_N({\bf S}_0^{-1}{\bf m}_0 + \beta{\bf \Phi}^T{\bf t}) \\ {\bf S}_N^{-1} &= {\bf S}_0^{-1} + \beta{\bf \Phi}^T{\bf \Phi} \end{align}

Then, he continues by saying that

If we consider an infinitely broad prior $${\bf S}_0 = \alpha^{-1}{\bf I}$$ with $$\alpha \to 0$$, the mean $${\bf m}_N$$ of the posterior distribution reduces to the maximum likelihood value $${\bf w}_{\text{ML}}$$

Where $${\bf w}_{\text{ML}} =({\bf \Phi}^T{\bf \Phi})^{-1}{\bf \Phi}{\bf t}$$.

In doing so, I arrive at $${\bf m}_N = {\bf m}_0 + \alpha^{-1}\beta{\bf \Phi}^T{\bf t} + \alpha\beta^{-1}({\bf \Phi}^T{\bf \Phi})^{-1}{\bf m}_0 + ({\bf \Phi}^T{\bf \Phi})^{-1}{\bf \Phi}{\bf t}$$

For which the only way to get rid of $${\bf m}_0$$ if it is a zero vector. Thus, we should consider a zero mean isotropic gaussian as a prior. In doing so I arrive at

$${\bf m}_N = \alpha^{-1}\beta{\bf \Phi}^T{\bf t} + ({\bf \Phi}^T{\bf \Phi})^{-1}{\bf \Phi}{\bf t}$$

Finally, if I take the limit $$\alpha \to 0$$ in $${\bf m}_0$$, then $$\lim_{\alpha \to 0^+} 1/\alpha = \infty$$ and $$\lim_{\alpha \to 0^-} 1/\alpha = -\infty$$, which does not converge and cannot reduced $${\bf m}_N$$ to $${\bf w}_{\text{ML}}$$. Since we want an infinitely broad prior, $${\bf S}_0$$ has to be $${\bf S}_0 = \alpha^{-1}{\bf I}$$ with $$\alpha \to 0$$.

What's the argument I need to conclude that $${\bf m}_N$$ does indeed converge to $${\bf w}_{\text{ML}}$$?

From the formulas for $$\mathbf{m_N}$$ and $$\mathbf{S_N}^{-1}$$, I arrive at: $$\mathbf{S_N^{-1} m_N=\alpha I\ m_0}+\beta\mathbf{\Phi^Tt}\rightarrow (\alpha\bf I + \beta{\bf \Phi}^T{\bf \Phi})\mathbf{m_N=\alpha\bf I\ m_0}+\beta\mathbf{\Phi^Tt}$$ Taking $$\alpha\rightarrow 0$$ in both sides of the equation leaves us with the following: $$\beta\ {\bf \Phi}^T{\bf \Phi}\ \mathbf{m_N}=\beta\ \mathbf{\Phi^Tt}\rightarrow \bf m_N=({\bf \Phi^T\Phi})^{-1}\bf \Phi^T\bf t$$