How can I derive the posterior distribution of a likelihood function $N({\bf{t}}|\phi(x)^{T}{\bf{w}}, \beta^{-1})$ and a prior $N({\bf{w}}|{\bf{m_{0}}}, \beta^{-1}{\bf{S_{0}}})Gam(\beta, a_{0},b_{0})$?

I'd like to use something similar to the general relations for conditional distributions in the normal case:

$$p(x|y) \propto p(y|x)p(x) = N(y|Ax+b, L^{-1})N(x|\mu, \Lambda^{-1})$$

which gives the result of:

$$p(x|y) \propto N(x|\Sigma\{A^{T}L(y-b)+\Lambda \mu\}, \Sigma)$$

However, this doesn't work with a normal-gamma distribution as a prior, and I'm not sure if there is such equivalent relation.

Any thoughts about how to approach this?


I assume you are trying to conduct Bayesian linear regression using a conjugate prior. The calculations are explained in the wikipedia entry Bayesian linear regression.

The only difference is that you are working with the precision $\beta$ instead of the variance $\sigma^2$, which are equivalent up to a change of variable.

| cite | improve this answer | |
  • $\begingroup$ Thanks. There is another difference in the use of the inverse Gamma distribution instead of the Gamma distribution, although I don't think it will create issue. Let me go through that proof to make sure. By the way, is there a set of general formulas for conditional distributions in the way I described above? $\endgroup$ – Robert Smith Apr 3 '13 at 20:53
  • $\begingroup$ Actually the model you are describing is exactly the same as the one described in the wikipedia entry since $\sigma^2 = \beta^{-1}$ (recall that if $X$ is gamma distributed, then $1/X$ is inverse-gamma distributed). You are then using the prior $p({\bf w},\beta)=p({\bf w}\vert\beta)p(\beta)$, which is a hierarchical prior. The posterior is $p({\bf w},\beta \vert {\bf t}) = N(...)p({\bf w},\beta)$. Then, yes, it is analogous to the normal-gamma case. $\endgroup$ – Sastre Apr 3 '13 at 20:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.