My question concerns Gibbs sampling. Suppose that I have three unknown quantities, $\mu, \sigma^2$ and $c$. I have given prior information and I have given the likelihood which allows me to compute the posterior $g(\mu, \sigma^2, c \ | \ \mbox{data})$. Now, I want to write a Gibbs sampler to generate from the posterior. To that end, the exercise says to consider the conditional distributions $[c \ | \ \mu, \sigma^2]$ and $[\mu, \sigma^2 \ | \ c]$.
Using the posterior I can easily find the corresponding distribution for $[c | \ \mu, \sigma^2]$. However, the conditional density $[\mu, \sigma^2 \ | \ c]$ does not have a known functional form. Therefore, my initial idea was to split it as follows: $$g[\mu, \sigma^2 \ | \ c] = g(\mu \ | \ \sigma^2,c) \underbrace{g(\sigma^2 \ | c)}_{(A)},$$ where $(A)$ can be found by integrating $g(\mu, \sigma^2 \ | c)$ w.r.t $\mu$. This gives me some nice distributions to work with, however, I do not have a dependence with $\mu$ anymore in $(A)$ since it has been integrated out. Therefore, I don't think this is the correct way. Instead, to write the Gibbs sampler I should consider the following conditional densities:
- $g(c \ | \ \mu, \sigma^2)$
- $g(\mu \ | \ c, \sigma^2)$
- $g(\sigma^2 \ | \mu, c)$.
Question: is the first idea wrong? I guess we can use it but it is not a Gibbs sampler then?