0
$\begingroup$

I'm very new to both Gibbs sampling and LDA. Currently, I'm trying to understand the collapsed Gibbs sampling method in LDA. However, I'm quit confused about the whole method when reviewing the algorithm. Here are my confusions and I badly need some guidance.

For short, let $W$ be the set of all the words in the corpus, $Z$ is the set of topics respect to them, and $\theta$ is the set of tunable hyperparameters. In LDA, the algorithm seems to try to sample from the distribution $p(Z | W; \theta)$. After the burning period, we get one sample $Z^\star \in p(Z | W; \theta)$. Am I right? If yes, why does the Gibbs sampling method work since there is only one sample $Z^\star$ that is sampled from the distribution? If I'm wrong, I badly want some detailed explainations.

The question may be naive, but I really need help.

$\endgroup$

1 Answer 1

0
$\begingroup$

There isn't just one $Z^\star$ sample you get from the run. Generally what one does is to run the algorithm for a large number of iterations, after the burn in period (first subset of the iterations), you collect samples, $Z^\star$, at regular intervals. Once you have decided that your algorithm has converged, you now have a distribution of $Z^\star$'s whose normalized counts should in principle follow $p\left(Z|W;\theta\right)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.