I'm very new to both Gibbs sampling and LDA. Currently, I'm trying to understand the collapsed Gibbs sampling method in LDA. However, I'm quit confused about the whole method when reviewing the algorithm. Here are my confusions and I badly need some guidance.
For short, let $W$ be the set of all the words in the corpus, $Z$ is the set of topics respect to them, and $\theta$ is the set of tunable hyperparameters. In LDA, the algorithm seems to try to sample from the distribution $p(Z | W; \theta)$. After the burning period, we get one sample $Z^\star \in p(Z | W; \theta)$. Am I right? If yes, why does the Gibbs sampling method work since there is only one sample $Z^\star$ that is sampled from the distribution? If I'm wrong, I badly want some detailed explainations.
The question may be naive, but I really need help.