I've been researching Bayesian linear regression. I'm confused about the following example.

Given the model:

$${\bf y} = {\bf \beta}{\bf X} + \bf{\epsilon} $$

Assuming that ${\bf \epsilon} \sim N(0, \phi I)$ and $p(\beta, \phi) \propto \frac{1}{\phi}$, we have:

$$p(\beta|\phi, {\bf y}) \sim N(({\bf X}^{\text{T}}{\bf X})^{-1}{\bf X}^{\text{T}}{\bf y}, \phi ({\bf X}^{\text{T}}{\bf X})^{-1})$$

How was $p(\beta|\phi, {\bf y})$ determined?


1 Answer 1


Your final formula is missing a left parenthesis.

This is a standard problem that requires no difficult work. The wikipedia page on Bayesian regression solves a harder problem; you should be able to use the same trick (which is basically just a form of completing the square, since you want it in terms of $(\beta - m)' V^{-1} (\beta - m)$ for some $m$ and $V$), with fewer terms to worry about. That is, you get to something like this:

\begin{align}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)^{\rm T}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)&= (\boldsymbol\beta - \hat{\boldsymbol\beta})^{\rm T}(\mathbf{X}^{\rm T}\mathbf{X})(\boldsymbol\beta - \hat{\boldsymbol\beta}) + \mathbf{S} \end{align}


\begin{align}\mathbf{S} = (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})^{\rm T}(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})\end{align}

in the exponent.

See also the references at the wikipedia article.

If this is for some subject, please mark it as homework.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.