# How does LDA (Latent Dirichlet Allocation) assign a topic-distribution to a new document?

I am new to topic modeling and read about LDA and NMF (Non-negative Matrix Factorization). I understand the training process work. Let's say I have 100 documents and I want to train an LDA for these documents with 10 topics. However, I don't really understand how does this model assign topic to an unseen document?

I used Gensim. After training, I have an LDA trained model and a dictionary with most frequent words. Let's say, I have an unseen new document with the following text:

This is just a test text about topic modeling and LDA.


Can someone explain step by step how a topic distribution is assigned to this new document in terms of algorithmic steps? The same goes for NMF method.

• By the context, I understand that LDA refers to Latent Dirichlet Allocation, but please clarify this in the question. Also include the full name for Non-negative Matrix Factorization. – Daniel López Jan 29 '18 at 14:37
• The Bayes decision rule of assigning topics to new documents depends on the loss function. – Łukasz Grad Jan 29 '18 at 14:49
• LDA does not assign topics to documents, it assigns topics to words and topic-distributions to documents. – guy Jan 29 '18 at 15:23
• @guy I should have explicitly specified that. I meant topic distribution. – nickg Jan 29 '18 at 15:25
• The topic distribution represented as a point on the $n_{topic}$-dimensional simplex, and is inferred by looking at the posterior under a Dirichlet prior. If we were to use, say, a Gibbs sampler, the topic distribution would be updated across iterations by sampling from the associated full conditional, which by conjugacy is another Dirichlet. – guy Jan 29 '18 at 17:23

What you should actually do is run inference (training) on the new set of documents (the old ones and the new ones together). A short-cut that estimates this well is applying Gibbs sampling only to the new documents while using the data obtained during training unchanged, as described by @SheldonCooper in Topic prediction using latent Dirichlet allocation.

As already mentioned in the comments, topics are assigned to individual words in a document. The Gibbs Sampler for latent Dirichlet allocation may give some insight into this. During training, the goal is to infer topics and per-document topic proportions by repeatedly sampling topic assignments,

$$P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} ) \propto \frac{n^{(w_i)}_{-i,j}+\beta}{n^{(.)}_{-i,j}+W\beta} \times \frac{n^{(d_i)}_{-i,j}+\alpha}{n^{(d_i)}_{-i,.}+T\alpha}$$

$P(z_i=j \mid \textbf{z}_{-i} , \textbf{w} )$ refers to the probability of assigning topic $j$ to $i^{th}$ word, given all other assignments.

After convergence, the following can be estimated easily,

$$\hat{\phi}_j^{(w)} = \frac{n_j^{(w)}+\beta}{n_j^{(.)}+W\beta}$$

$$\hat{\theta}_j^{(d)} = \frac{n_j^{(d)}+\alpha}{n_.^{(d)}+T\alpha}$$

where, $\hat{\phi}_j^{(w)}$ is the probability of $w^{th}$ word in the $j^{th}$ topic and $\hat{\theta}_j^{(d)}$ is the probability of the $j^{th}$ topic in the $d^{th}$ document.

At testing time, the (un-normalized) posterior probability of assigning a word to topic $j$ can be easily estimated as follows,

\begin{align} P(z=j \mid w, \boldsymbol{\phi}, \boldsymbol{\theta}) &\propto P(w \mid z=j, \hat{\boldsymbol{\phi}}_j) \times P(z=j \mid \hat{\boldsymbol{\theta}}^{(d)})\\ &\propto \hat{\phi}_j^{(w)} \hat{\theta}_j^{(d)} \end{align}

This way one can find the probability of assigning each topic to a word, and select the topic with maximum probability i.e. find the maximum a posteriori estimate.

• The problem with this answer is that the question referred to new documents, and in your posterior calculation you use the topic distribution for a specific document d, which you do not have any estimate on when d is a new document. – emem Feb 20 '18 at 13:13