# Gibbs sampling and conditional distribution

I need to simulate the posterior distribution of intraclass correlation coefficient $\pi(\rho|y)$ where $y$ is the data set and $\rho=\frac{\sigma_a^2}{\sigma_a^2+\sigma_e^2}$ with $\sigma^2_a\sim IG(\beta,1)$ , $\sigma_e^2\sim IG(\alpha,1)$, $\rho\sim Beta(\alpha,\beta)$ where IG stands for inverse Gamma distribution.

Taking the random effect model $$Y_{ij}=\mu+\alpha_i+e_{ij};i=1,\dots,n;j=1,\dots,m$$ where $\mu\sim N(0,1e+06)$,$\alpha_i\sim N(0,\sigma_a^2)$ and $e_{ij}\sim N(0,\sigma_e^2)$, the text say that I can simulate this posterior with the following information

$$\begin{cases} [\theta_i|\mu,\sigma_a^2]\sim N(\mu,\sigma_a^2);\theta_i=\mu+\alpha_i\ i=1,\dots,n\\ [e_{ij}|\sigma_e^2]\sim N(0,\sigma_e^2)\\ [y_{ij}|\theta_i,\sigma_e^2]\sim N(\theta_i,\sigma_e^2)\\ [\theta|y,\mu,\sigma_a^2,\sigma_e^2]\sim N_n\Big(\frac{m\sigma_a^2}{m\sigma_a^2+\sigma_e^2}\overline{y}+\frac{\sigma_e^2}{m\sigma_a^2+\sigma_e^2}\mu\textbf{1}_n,\frac{\sigma_a^2\sigma_e^2}{m\sigma_a^2+\sigma_e^2}\mathbb{I}_n\Big)\\ [\mu|y,\theta,\sigma_a^2,\sigma_e^2]=[\mu|\sigma_a^2,\theta]\sim N\Big(\frac{n(1e+06)}{n(1e+06)+\sigma_a^2}\overline{\theta},\frac{n(1e+06)}{n(1e+06)+\sigma_a^2}\frac{\sigma_a^2}{n}\Big)\\ [\sigma_e^2|y,\mu,\theta,\sigma_a^2]=[\sigma_e^2|y,\theta]\sim IG\Big(\alpha+\frac{nm}{2},1+\frac{\sum_{j=1}^m\sum_{i=1}^n(y_{ij}-\theta_i)^2}{2}\Big)\\ [\sigma_a^2|y,\mu,\theta,\sigma_e^2]=[\sigma_a^2|\mu,\theta]\sim IG\Big(\beta+\frac{n}{2},1+\frac{\sum_{i=1}^n(\theta_i-\mu)^2}{2}\Big)\\ \overline{\theta}=\frac{1}{n}\sum_{i=1}^n \theta_i , y_i=\frac{1}{m}\sum_{i=1}^m y_{ij} \end{cases}$$

Obs: The reference I used as base is here sampling-Based Approaches to Calculating Marginal Densitie

I know that if I have $[\sigma_e^2|y,\mu,\theta,\sigma_a^2]$ and $[\sigma_a^2|y,\mu,\theta,\sigma_e^2]$ I can get the distribution of $\rho$. What I want to know is

How they get the distributions of $[\sigma_e^2|y,\mu,\theta,\sigma_a^2]$ and $[\sigma_a^2|y,\mu,\theta,\sigma_e^2]$ using the conditional distributions?

• @Xi'an Thanks, I want to know how they get the sample of $\sigma_a^2$ and $\sigma_e^2$ using the conditional distributions. – user72621 Jun 5 '16 at 19:16
• If you do not know about Gibbs sampling, you should invest into an introduction to the topic. See for instance this mini-lecture. Or this longer one about MCMC. About minute 34 for reaching Gibbs sampling. – Xi'an Jun 5 '16 at 20:10
• @Xi'an The joint distribution $[y,\mu,\theta,\sigma_a^2,\sigma_e^2]=[y|\theta,\sigma_e^2]*[\theta|\mu,\sigma_a^2]*[\mu]*[\sigma_a^2]*[\sigma_e^2]$, I don't understand well the marginalization process to get the conditionals of $\sigma_a^2$ and $\sigma_e^2$ from the joint distribution – user72621 Jun 5 '16 at 21:54
• Pragakham -- please amend your question, which now seems to be about deriving conditional distributions – Glen_b Jun 6 '16 at 5:50

When you consider the joint distribution of the model $$f(y,\mu,\alpha,\sigma_a^2,\sigma_e^2)\propto\prod_{i=1}^n\prod_{j=1}^m \sigma_e^{-1}\exp\{-(y_{ij}-\mu-\alpha_i)^2/2\sigma_e^2\}\times\prod_{i=1}^n\sigma_a^{-1}\exp\{-\alpha_i^2/2\sigma_a^2\}\times\pi(\mu)\times\sigma_a^{-2(\beta+1)}\exp\{-1/\sigma_a^2\}\times\sigma_e^{-2(\alpha+1)}\exp\{-1/\sigma_e^2\}$$ the conditional of $\sigma_a^2$, $f(\sigma_a^2|y,\mu,\alpha,\sigma_e^2)$ is proportional to $f(y,\mu,\alpha,\sigma_a^2,\sigma_e^2)$ as a function of $\sigma_a^2$. Hence you can get rid of all multiplicative terms that do no involve $\sigma_a$: $$f(\sigma_a^2|y,\mu,\alpha,\sigma_e^2)\propto\prod_{i=1}^n\sigma_a^{-1}\exp\{-\alpha_i^2/2\sigma_a^2\}\times\sigma_a^{-2(\beta+1)}\exp\{-1/\sigma_a^2\}$$ which factorises as $$f(\sigma_a^2|y,\mu,\alpha,\sigma_e^2)\propto\sigma_a^{-n-2\beta-2}\exp\left\{-\sigma_a^{-2}\left[1+\frac{1}{2}\sum_{i=1}^n\alpha_i^2\right]\right\}$$ which you can identify as an $IG(\beta+n/2,1+\frac{1}{2}\sum_{i=1}^n\alpha_i^2)$ density.
A similar argument leads to the conditional distribution of $\sigma_e^2$ as being another $IG$ distribution that does not depend on $\sigma_a^2$. Note however that the ratio $$\rho=\dfrac{\sigma_a^2}{\sigma_a^2+\sigma_e^2}$$ does not enjoy a Beta distribution any longer, since the scales of the two $IG$ distributions differ. It is therefore doubly necessary to simulate both $\sigma_a$ and $\sigma_e$ since (a) they are used to produce simulations for the other parameters and vice-versa, and (b) $\rho$ cannot be simulated directly.
Remember that, when you use Gibbs sampling, you produce a sample of random variables simulated from the target distribution. In your case, the Gibbs sampler will produce a sample of $(σ^2_a,σ^2_e)$, hence a sample of ρ's. The empirical distribution function based on that sample is a converging approximation to the true posterior distribution of ρ.
• Thanks for this answer, so to get the full conditionals from the joint pdf , I just need to ignore the terms that not involve the conditional that you want? The use o Gibbs sampling in this case, is because the joint distribution does not have a analytical expression right? The posterior distribution of $\rho$ still is Beta right? I was breaking my head, trying to understand why I need simulation if the conditionals of $\sigma_a^2$ and $\sigma_e^2$ are IG. – user72621 Jun 7 '16 at 14:25