# If the Gibbs Sampler assumes knowledge of full conditionals — why can't we just solve for the full joint density and avoid Gibbs?

Suppose that $y_1, y_2$ are data drawn from a density function $f$ with parameters $\theta_1, \theta_2$ which are unknown. Suppose I applied some prior on $\theta_1, \theta_2$. Then, the Gibbs sampler states that if we know the FULL conditional distributions:

$$p(\theta_1|\theta_2, y) \ \ \ \text{and} \ \ \ p(\theta_2|\theta_1, y)$$

then we may draw from each and be able to approximate draws from the joint distribution $p(\theta_1, \theta_2)$, which is assumed to be hard to find (which is why we do Gibbs here to begin with).

We may rewrite the above as:

$$p(\theta_1|\theta_2, y) = \frac{p(\theta_1,\theta_2, y)}{p(\theta_2, y)} \ \ \ \text{and} \ \ \ p(\theta_2|\theta_1, y) = \frac{p(\theta_2,\theta_1, y)}{p(\theta_1, y)}$$

Now, I assume that we know $p(\theta_1, y)$ and $p(\theta_2, y)$ from the prior specification. Then, if we know $p(\theta_1, y)$ and $p(\theta_1|\theta_2, y)$, don't we also know $p(\theta_1,\theta_2, y)$?

What exactly is the Gibb's sampler doing here?

Mathematically, this is correct: once you know $$p(\theta_1|\theta_2, y) \ \ \ \text{and} \ \ \ p(\theta_2|\theta_1, y)$$ you can derive the joint posterior distribution as $$p(\theta_1,\theta_2|y) = \dfrac{p(\theta_1|\theta_2,y)}{\int p(\theta_1|\theta_2,y)\big/p(\theta_2|\theta_1, y)\,\text{d}\theta_1}$$ [This is Theorem 9.3 in our book.]
However the reason for running Gibbs sampling is that/when this expression is not available in closed form and hence cannot be simulated directly. In the event one knows $$p(\theta_1,y)$$ or $$p(\theta_2,y)$$ in closed form and can simulate $$\theta_1$$ or $$\theta_2$$ from the former or the latter, resp., then one does not require Gibbs sampling as direct simulation becomes feasible.
• So you're basically saying that even if I can compute $p(\theta_1,\theta_2, y)$ from the full conditionals, to get to $p(\theta_1,\theta_2|y)$, I still need the $p(y)$ on the denominator, which is the normalizing constant and hence whose closed form is the deciding factor in whether to use Gibbs or not? – user321627 Feb 16 '17 at 6:45
• I see, so $\int p(\theta_1|\theta_2,y)\big/p(\theta_2|\theta_1, y)\,\text{d}\theta_1 = \frac{1}{p(\theta_2,y)}\int p(\theta_1, y)d\theta_1 = \frac{p(y)}{p(\theta_2,y)}$. For the closed form and simulation requirement, is to facilitate the calculation of the integral part $\int p(\theta_1, y)d\theta_1$ or the fraction part $\frac{1}{p(\theta_2,y)}$? Thank you! – user321627 Feb 16 '17 at 9:23