Gibbs sampling, what to use? My question concerns Gibbs sampling. Suppose that I have three unknown quantities, $\mu, \sigma^2$ and $c$. I have given prior information and I have given the likelihood which allows me to compute the posterior $g(\mu, \sigma^2, c \ | \ \mbox{data})$. Now, I want to write a Gibbs sampler to generate from the posterior. To that end, the exercise says to consider the conditional distributions $[c \ | \ \mu, \sigma^2]$ and $[\mu, \sigma^2 \ | \ c]$. 
Using the posterior I can easily find the corresponding distribution for $[c | \ \mu, \sigma^2]$. However, the conditional density $[\mu, \sigma^2 \ | \ c]$ does not have a known functional form. Therefore, my initial idea was to split it as follows:
$$g[\mu, \sigma^2 \ | \ c] = g(\mu \ | \ \sigma^2,c) \underbrace{g(\sigma^2 \ | c)}_{(A)},$$
where $(A)$ can be found by integrating $g(\mu, \sigma^2 \ | c)$ w.r.t $\mu$. This gives me some nice distributions to work with, however, I do not have a dependence with $\mu$ anymore in $(A)$ since it has been integrated out. Therefore, I don't think this is the correct way. Instead, to write the Gibbs sampler I should consider the following conditional densities:


*

*$g(c \ | \ \mu, \sigma^2)$

*$g(\mu \ | \ c, \sigma^2)$

*$g(\sigma^2 \ | \mu, c)$. 


Question: is the first idea wrong? I guess we can use it but it is not a Gibbs sampler then? 
 A: The exercise asked you implement a Gibbs sampler using the full conditionals of $[c \mid \mu, \sigma^2]$ and $[\mu, \sigma^2 \mid c]$. This is a valid Gibbs sampler and has the special name of Blocked Gibbs Sampler, since $\mu$ and $\sigma^2$ are treated as one variable using their joint conditional distribution. To see this think of $\theta = (\mu, \sigma^2)$. Then the blocked Gibbs sampler would look a usual Gibbs sampler with the conditionals being $[c \mid \theta]$ and $[\theta \mid c]$. The tricky part is, how do you find $[\theta \mid c]$? Your idea of using
$$g[\theta \mid c] = g[\mu, \sigma^2 \mid c] = g[\mu \mid \sigma^2, c]\,  \underbrace{g[\sigma^2 \mid c]}_{(A)}$$
is absolutely the right idea and is correct. With these distributions, your blocked Gibbs sampler will update as follows. Let $c_0, \mu_0$ and $\sigma^2_0$ will be the starting values. 


*

*$c_1 \sim g[c \mid \mu_0, \sigma^2_0]$

*$\sigma^2_1 \sim g[\sigma^2 \mid c_1]$

*$\mu_1 \sim g[\mu \mid \sigma^2_1, c_1]$


and so on.
