How can I derive the posterior distribution of a likelihood function $N({\bf{t}}|\phi(x)^{T}{\bf{w}}, \beta^{-1})$ and a prior $N({\bf{w}}|{\bf{m_{0}}}, \beta^{-1}{\bf{S_{0}}})Gam(\beta, a_{0},b_{0})$?

I'd like to use something similar to the general relations for conditional distributions in the normal case:

$$p(x|y) \propto p(y|x)p(x) = N(y|Ax+b, L^{-1})N(x|\mu, \Lambda^{-1})$$

which gives the result of:

$$p(x|y) \propto N(x|\Sigma\{A^{T}L(y-b)+\Lambda \mu\}, \Sigma)$$

However, this doesn't work with a normal-gamma distribution as a prior, and I'm not sure if there is such equivalent relation.

Any thoughts about how to approach this?


1 Answer 1


I assume you are trying to conduct Bayesian linear regression using a conjugate prior. The calculations are explained in the wikipedia entry Bayesian linear regression.

The only difference is that you are working with the precision $\beta$ instead of the variance $\sigma^2$, which are equivalent up to a change of variable.

  • $\begingroup$ Thanks. There is another difference in the use of the inverse Gamma distribution instead of the Gamma distribution, although I don't think it will create issue. Let me go through that proof to make sure. By the way, is there a set of general formulas for conditional distributions in the way I described above? $\endgroup$
    – r_31415
    Apr 3, 2013 at 20:53
  • $\begingroup$ Actually the model you are describing is exactly the same as the one described in the wikipedia entry since $\sigma^2 = \beta^{-1}$ (recall that if $X$ is gamma distributed, then $1/X$ is inverse-gamma distributed). You are then using the prior $p({\bf w},\beta)=p({\bf w}\vert\beta)p(\beta)$, which is a hierarchical prior. The posterior is $p({\bf w},\beta \vert {\bf t}) = N(...)p({\bf w},\beta)$. Then, yes, it is analogous to the normal-gamma case. $\endgroup$
    – Sastre
    Apr 3, 2013 at 20:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.