# In LDA, after collapsed Gibbs sampling, how to estimate values of other latent variables?

I watched a video on coursera, everything went well until the following slide around 12'50''.

I read it in other papers that to estimate latent variables say $\Phi$ we can draw a sample $Z'$ of the full collection by collapsed Gibbs sampling and take $p(\Phi|W,Z')$ as an approximation.

This video says we can use $p(\Phi|W,\hat{Z})$ where $\hat{Z}$ is an average of a few samples.

My question is, does it make sense to average the samples of $Z$? If I understood correctly $z_{dn}$ represent the topic index that is responsible for the $n$th word in the $d$th document. But for two independent samples, the indices of the topics are not guaranteed to be the same.

• You should write the equations, and not post a screenshot, here stats.stackexchange.com/questions/346302/… @Glen_b explains why Commented May 15, 2018 at 13:55
• @niandra82 thanks but I dont see how this is necessary here, the screenshot looks rather clear to me, even if it's for users who rely on screen readers there's link to the video with a lecturer who explains it better than a screen reader. Commented May 15, 2018 at 15:48
• You have a (lot) higher reputation than me, so you are probably right. But I don't see why I should see the entire video to understand what $\theta$ or $Z$ are, for example. If you write the equations, you may be forced to explain what they represent. Just my opinion Commented May 15, 2018 at 16:10
• @niandra82 if you're saying the equations need more explanation then that's a different problem which can not be improved by simply retyping all the equations, surely I'll try to add more explanation if i have time. Those who are familiar with lda should basically know what I'm asking only by the title so the screenshot is there to provide a common page of notations. Commented May 16, 2018 at 2:44

The LDA has the following three latent variables:

1. $\phi_t$ - Topics - which are $W$ dimensional discrete distributions.
2. $\theta_d$ - per document topic proportions - which are $T$ dimensional discrete distributions.
3. $z_{d,n}$ which is the topic association of the $n$th word in the $d$th document ($w_{d,n}$). $z_{d,n} \in \{1, 2, ... T\}$.

Using collapsed Gibbs sampling, the topic allocations $z_{d,n}$ for words $w_{d,n}$ are sampled. Using these topic allocations, $\phi_t$ and $\theta_d$ can be estimated as follows:

$$\phi_t(w) = \frac{n_j^{(w)}+\beta}{n_j^{(.)}+W\beta} \\ \theta_d(t) = \frac{n_j^{(d)}+\alpha}{n_.^{(d)}+T\alpha}$$

where,

• $n_j^{(w)}:$ number of times $w^{th}$ word was assigned to topic $j$
• $n_j^{(.)}:$ total number of words assigned to topic $j$
• $n_j^{(d)}:$ number of times topic $j$ was assigned in $d^{th}$ document
• $n_.^{(d)}:$ total number of topics assigned in $d^{th}$ document
• $T:$ number of topics
• $W:$ number of words in vocabulary
• $\alpha, \beta:$ Dirichlet hyperparameters

Reference: Griffiths et. al., (2004)

• Hi thank you that makes perfect sense, but actually I want to asked about the method in the video, how they average different samples of Z? If I understood correctly all the $n$s needed are computed based on one sample of the full collection $Z\in R^{W}$, the video mentioned about averaging different $Z$s and that's where I'm confused. Commented May 18, 2018 at 3:11
• @dontloo you are correct that all the $n$'s are based on the topic assignments for the entire collection of documents. You don't really average over different $Z$'s. If you look at the implementation of the collapsed Gibbs sampler by the authors, nowhere do they average over $Z$'s. Commented May 18, 2018 at 15:27