How does Gibbs sampling work in Latent Dirichlet Allocation? It seems that only one sample is sampled from the distribution of corpus topics

Question

I'm very new to both Gibbs sampling and LDA. Currently, I'm trying to understand the collapsed Gibbs sampling method in LDA. However, I'm quit confused about the whole method when reviewing the algorithm. Here are my confusions and I badly need some guidance.

For short, let $W$ be the set of all the words in the corpus, $Z$ is the set of topics respect to them, and $\theta$ is the set of tunable hyperparameters. In LDA, the algorithm seems to try to sample from the distribution $p(Z | W; \theta)$. After the burning period, we get one sample $Z^\star \in p(Z | W; \theta)$. Am I right? If yes, why does the Gibbs sampling method work since there is only one sample $Z^\star$ that is sampled from the distribution? If I'm wrong, I badly want some detailed explainations.

The question may be naive, but I really need help.

Answer 1

There isn't just one $Z^\star$ sample you get from the run. Generally what one does is to run the algorithm for a large number of iterations, after the burn in period (first subset of the iterations), you collect samples, $Z^\star$, at regular intervals. Once you have decided that your algorithm has converged, you now have a distribution of $Z^\star$'s whose normalized counts should in principle follow $p\left(Z|W;\theta\right)$.