How to get the topic coverage for a document using LDA?

Question

I have three distributions of topic obtained by Latent Dirichlet allocation:

• theme_1 = {cat*0.7, dog*0.2, pet*0.1},
• theme_2 = {salad*0.5, fish*0.3, chicken*0.2},
• theme_3 = {cuisine*0.4, food*0.3, tomato*0.3},

I need to calculate the coverage of each item in a document having a word count vector for example:

• d1 = {Dog: 4, Fish: 6, Cooking: 2: Tomato: 0, Chicken: 5, Meal: 2}

to result in a topic vector instead of the word count vector in this way:

• d1 = {theme_1, theme_2, theme_3}.

I was calculating this way:

• theme_1 in d1 = probability(dog in theme_1) * # Occurrences(dog in d1)

Because only "dog" occurs in the topic "theme_1" and in the document and so on.

Is it the right way to do it? Because I also understand that the coverage of topics in a document must be equal to 1.

Answer 1

As there are SO questions and solutions for this problem in both R and Python, I assume you came here to ask for the mathematical/algorithmic explanation how these tools generate the per-document topic distributions.

Using Blei's typical nomenclature, we let $\theta_{d,j}$ be the proportion of topic $j$ in document $d$. As we know from Blei, the LDA model is defined as the following joint probability:

$$ p( \overrightarrow{\beta}, \overrightarrow{\theta}, \overrightarrow{z}, \overrightarrow{w} ; \alpha, \eta ) = \left( \prod_i p( \beta_i | \eta ) \right) \left( \prod_d p( \theta_d | \alpha) \prod_n p(z_{d,n} | \theta_d ) p( w_{d,n} | \overrightarrow{\beta}, z_{d,n}) \right) $$

With $\alpha$ and $\eta$ being the two Dirichlet priors (for documents and topics, respectively).

Therefore, the posterior estimate for a specific topic and document is - e.g., if approximated by stochastic variational inference via (re-) running the inference (algorithm) on the (final) model - essentially:

$$ \theta_{d,j} = \frac{ C_{d,j} + \alpha_j } { \sum_k C_{d,k} + \alpha_k } $$

Where $C$ is the document-topic assignment (counts) matrix. That is, the raw result of a single inference step (also see gensim's LdaModel.inference). Note that due to sampling from the Dirichlet prior during the inference, your results will vary ever so slightly on each run. But if your model is good, the variations should be small (so this is a good way to estimate model quality).