Gibbs sampling from posterior distribution using R

Question

New to MCMC.

I have a model, saying $$Y_i=\beta_0+\beta_1x_{i1}+\beta_2x_{i2}+\frac{e_i}{\sqrt{\mu}}$$ where $x_{ij}$ are fixed covariates, $e_i\sim N(0,1)$, $\beta_0$, $\beta_1$, $\beta_2$ and $\mu$ are unknown parameters, with the prior distribution for parameters $\beta_i\sim N(0,1)$ and $\mu\sim \Gamma(0.1,0.1)$

Now I need to implement a Gibbs sampling of $(\beta_0,\beta_1,\beta_2,\mu)$ procedure from the joint posterior distribution with R.

I did some calculation but not sure: is this the posterior distribution? $$ C\left(\frac{1}{\sqrt{2\pi}}e^{-\frac{\beta_0^2}{2}}\right)\left(\frac{1}{\sqrt{2\pi}}e^{-\frac{\beta_1^2}{2}}\right)\left(\frac{1}{\sqrt{2\pi}}e^{-\frac{\beta_2^2}{2}}\right)\mu^{0.1-1}e^{-0.1\mu} $$ $$ \qquad\qquad \times\prod_{i=1}^n\frac{\sqrt{\mu}}{\sqrt{2\pi}}e^{-\frac{\mu}{2}[y_i-(\beta_0+\beta_1x_{i1}+\beta_2x_{i2})]^2} $$

But I still don't know how to derive the posterior distribution, neither do I know how to do the sampling with R. Can anyone give me some references? Thanks very much.

Answer 1

Your posterior is correct (I edited a missing $\sqrt{}$). Because you are using conjugate priors you actually do not need Gibbs sampling, you can derive exactly the posterior distribution. This is shown for instance in our book, Bayesian Core, where the entire chapter 3 is dedicated to Gaussian linear regression.

If you really want to run a Gibbs sampler (is this an homework?!), the above posterior is all you need. From there, you extract the terms that depend on $\beta_0$, $$ \pi(\beta_0|\beta_{-0},x,\mu)\propto \exp\left[-\beta_0^2/2-\mu\sum_i \{y_i−(β_0+β_1x_{i1}+β_2x_{i2})\}^2/2\right] $$ and this gives you a Gaussian distribution with parameters depending on the data and the other parameters. Then you do the same for $\beta_1$, and $\beta_2$, again ending up with two Gaussian distributions. Then for $\mu$, where the conditional distribution is then a Gamma distribution. You should find further details, if needed, in any Bayesian book, including Bayesian Core, Monte Carlo Statistical Methods, and Bayesian Data Analysis.