I am trying to solve the following problem.

If $y | \beta \sim N(X \beta, I_n)$ and $\beta \sim N(0, g^{-1}(X^t X)^{-1})$ for $g>0$. Find $ \pi(\beta|y)$ and show that $E(\beta|y)$ is a function of MLE of $\beta$.

My approach,

I proved that to maximize the likelihood, it is equivalent to minimizing the SSE, so the OLS is equal to the MLE.

I know normal is conjugate prior. So, $$\pi(\beta|y)= \frac{f(y|\beta)\pi(\beta)}{m(y)} \propto f(y|\beta)\pi(\beta) $$

$$ \pi(\beta|y) \propto \exp\{ - \frac{1}{2}(y-X\beta)^t (y-X\beta) - \frac{1}{2}\beta^t(g^{-1}(X^tX)^{-1})^{-1}\beta \} $$ $$ = \exp\{ - \frac{1}{2}( y^t y -2y^tX \beta+(1+g)\beta^t(X^tX)\beta ) \}$$ and I get stuck. I read about g-priors and I think the a posterior is $N(\frac{1}{1+g}\hat{\beta}, \frac{1}{1+g} (X^t X)^{-1})$ but I'm not sure.


You may consult any textbook on Bayesian econometrics (e.g.) to find the conditional posterior distribution for $\beta$ given $\sigma^2$, given "conjugate priors" for $\beta$ and $\sigma^2$, i.e., $$\pi \left( \beta ,\sigma ^{2}\right) =\pi \left( \beta |\sigma ^{2}\right)\pi \left( \sigma ^{2}\right) $$ with \begin{eqnarray} \beta |\sigma ^{2} &\sim &N\left( \beta _{0},\sigma ^{2}B_{0}\right) \\ \sigma ^{2} &\sim &IG\left( \alpha _{0}/2,\delta _{0}/2\right), \end{eqnarray} as \begin{eqnarray*} \beta |\left( \sigma^{2},y\right) &\sim& N\left( \beta _{1},\sigma^{2}B_{1}\right) \end{eqnarray*} where \begin{eqnarray*} B_{1} &=&\left( X^{\prime }X+B_{0}^{-1}\right) ^{-1} \\ \beta _{1} &=&B_{1}\left( X^{\prime }y+B_{0}^{-1}\beta _{0}\right) \end{eqnarray*} Here, it seems to be the case that $\sigma^2$ is known to be one, so will disappear from these expressions. Also, your prior mean is zero and $B_0=g^{-1}(X'X)^{-1}$.

Thus, \begin{eqnarray*} B_1 &=& (X'X + ((g\cdot X'X)^{-1})^{-1})^{-1}\\ & = &(X'X + g\cdot X'X)^{-1} = ((1+g)X'X)^{-1}\\ &=& 1/(1+g)(X'X)^{-1} \end{eqnarray*} and \begin{eqnarray*} \beta_1 &=& B_1(X'y+g\cdot X'X\beta_0)\\ & = & B_1 X'y = ((1+g)X'X)^{-1}X'y\\ & = & 1/(1+g)(X'X)^{-1}X'y = 1/(1+g)\hat\beta \end{eqnarray*} The part regarding the posterior mean should follow immediately.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.