I've been researching the use of Bayesian linear regression, but I've come to an example that I'm confused about.

Given the model:

$${\bf y} = {\bf \beta}{\bf X} + \bf{\epsilon} $$

Assuming that ${\bf \epsilon} \sim N(0, \phi I)$ and a $p(\beta, \phi) \propto \frac{1}{\phi}$,

How is $p(\beta|\phi, {\bf y})$ reached?

Where: $p(\beta|\phi, {\bf y}) \sim N({\bf X}^{\text{T}}{\bf X})^{-1}{\bf X}^{\text{T}}{\bf y}, \phi ({\bf X}^{\text{T}}{\bf X})^{-1})$.


Your final formula is missing a left parenthesis.

This is a standard problem that requires no difficult work. The wikipedia page on Bayesian regression solves a harder problem; you should be able to use the same trick (which is basically just a form of completing the square, since you want it in terms of $(\beta - m)' V^{-1} (\beta - m)$ for some $m$ and $V$), with fewer terms to worry about. That is, you get to something like this:

\begin{align}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)^{\rm T}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)&= (\boldsymbol\beta - \hat{\boldsymbol\beta})^{\rm T}(\mathbf{X}^{\rm T}\mathbf{X})(\boldsymbol\beta - \hat{\boldsymbol\beta}) + \mathbf{S} \end{align}


\begin{align}\mathbf{S} = (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})^{\rm T}(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})\end{align}

in the exponent.

See also the references at the wikipedia article.

If this is for some subject, please mark it as homework.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.