1
$\begingroup$

I am new to Gibbs sampling and sampling in general, so here is a basic question. I am reading this tutorial. Equation (40) is our complicated joint probability and equation (49) the less complicated conditional probability. What is the main obstacle in sampling from (40) that calls for using (49)? After all, for every value of the parameters and variables we can evaluate it using a computational softwar, say R, cannot we?

For completeness I write below formula (40) and (49) :

$\mathrm{P}(\mathbb{C},\mathbf{L},\theta_0,\theta_1;\pmb\mu) =\frac{\Gamma(\gamma_{\pi0}+\gamma_{\pi1})\Gamma(C_1+\gamma_{\pi1})\Gamma(C_0+\gamma_{\pi0})}{\Gamma(\gamma_{\pi1})\Gamma(\gamma_{\pi0}) \Gamma(N+\gamma_{\pi0}+\gamma_{\pi1})}\times\Pi_{i=1}^V\theta_{1,i}^{\mathcal{N}_{\mathbb{C}_1}(i)+\gamma_{\theta_i}-1}\Pi_{i=1}^V\theta_{0,i}^{\mathcal{N}_{\mathbb{C}_0}(i)+\gamma_{\theta_i}-1}.\;\;\;(40)$

And

$\mathrm{P}(\mathbf{L}_j=x|\mathbf{L}^{-j},\mathbb{C}^{-j},\theta;\pmb\mu)=\frac{C_x+\gamma_{\pi_x}-1}{N+\gamma_{\pi1}+\gamma_{\pi0}-1}\Pi_{i=1}^V\theta_{x,i}^{\mathbf{W}_{ji}}.\;\;\;\;\;(49)$

$\endgroup$
  • 2
    $\begingroup$ Please provide the details from the paper that avoid members of this forum to have to read the paper to understand your question. $\endgroup$ – Xi'an Nov 3 '16 at 21:16
  • 2
    $\begingroup$ To know the value of the density at any given parameter is not sufficient to derive a sample from that density, in practice. $\endgroup$ – Xi'an Nov 3 '16 at 21:19
  • $\begingroup$ You might find this article useful: medium.com/@aliaksei.mikhailiuk/… $\endgroup$ – Aliaksei Mikhailiuk Apr 27 at 15:46
1
$\begingroup$

I will start my answer by saying at first, as already Xi'an commented, that in order to sample the parameters you don't evaluate the distributions.

The distribution you have in equation (40) is the full probability model and you must sample its parameters.

In order to do so, one performs Gibbs sampling. Sampling from each conditional distribution of the parameters, if you concatenate them into a matrix it is equivalent as if you have sampled from the joint.

Also, notice that equation (49) is just the full conditional for the document labels. Sampling only from (49) is not the same as sampling from (40). So you also have to sample from the full conditionals of the other parameters. After a sweep of the sampler in all variables, you will have a sample from (40).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.