Why Gibbs sampling?

Question

I am new to Gibbs sampling and sampling in general, so here is a basic question. I am reading this tutorial. Equation (40) is our complicated joint probability and equation (49) the less complicated conditional probability. What is the main obstacle in sampling from (40) that calls for using (49)? After all, for every value of the parameters and variables we can evaluate it using a computational softwar, say R, cannot we?

For completeness I write below formula (40) and (49) :

$\mathrm{P}(\mathbb{C},\mathbf{L},\theta_0,\theta_1;\pmb\mu) =\frac{\Gamma(\gamma_{\pi0}+\gamma_{\pi1})\Gamma(C_1+\gamma_{\pi1})\Gamma(C_0+\gamma_{\pi0})}{\Gamma(\gamma_{\pi1})\Gamma(\gamma_{\pi0}) \Gamma(N+\gamma_{\pi0}+\gamma_{\pi1})}\times\Pi_{i=1}^V\theta_{1,i}^{\mathcal{N}_{\mathbb{C}_1}(i)+\gamma_{\theta_i}-1}\Pi_{i=1}^V\theta_{0,i}^{\mathcal{N}_{\mathbb{C}_0}(i)+\gamma_{\theta_i}-1}.\;\;\;(40)$

And

$\mathrm{P}(\mathbf{L}_j=x|\mathbf{L}^{-j},\mathbb{C}^{-j},\theta;\pmb\mu)=\frac{C_x+\gamma_{\pi_x}-1}{N+\gamma_{\pi1}+\gamma_{\pi0}-1}\Pi_{i=1}^V\theta_{x,i}^{\mathbf{W}_{ji}}.\;\;\;\;\;(49)$

Answer 1

I will start my answer by saying at first, as already Xi'an commented, that in order to sample the parameters you don't evaluate the distributions.

The distribution you have in equation (40) is the full probability model and you must sample its parameters.

In order to do so, one performs Gibbs sampling. Sampling from each conditional distribution of the parameters, if you concatenate them into a matrix it is equivalent as if you have sampled from the joint.

Also, notice that equation (49) is just the full conditional for the document labels. Sampling only from (49) is not the same as sampling from (40). So you also have to sample from the full conditionals of the other parameters. After a sweep of the sampler in all variables, you will have a sample from (40).