4
$\begingroup$

I'm interested to learn how I can use a trained LDA (Latent Dirichlet Allocation) model to make predictions on the topic proportion of a new, unseen document using Naive Bayes.

Let $z \in \{1, 2, ..., Z\}$ denote a particular topic (there's $Z$ topics in total) and $w_1, w_2, ..., w_n$ denote the $n$ words in the new document. Essentially, I'm trying to calculate the posterior probability for each topic $z$ conditional on having seen all the words in the question, i.e.,

$P(z|w_1, w_2, ..., w_n) = \frac{P(w_1, w_2, ..., w_n|z)\cdot P(z)}{P(w_1, w_2, ..., w_n)} = \frac{\prod_{i=1}^{n}P(w_i|z)\cdot P(z)}{P(w_1, w_2, ..., w_n)}$

where $P(w_1, w_2, ..., w_n|z) = \prod_{i=1}^{n}P(w_i|z)\cdot P(z)$ follows the Naive Bayes assumption of conditional independence between the words. My questions are:

  1. I understand that after training is complete, each word in each document in the training set will have its own topic assignment and that each document will have its own topic proportion/distribution. This is a result of repeated Gibbs sampling. Is it possible to make the same word-to-topic assignment in a new, unseen document? By using the "folding in" method described here I believe it's possible to achieve this, but I'm interested in if it's also possible through Naive Bayes. As I understand it, the posterior probability above gives you the probability of a given topic conditional on seeing all the words; it doesn't contain information on how to assign a topic to a single word.
  2. What kind of $P(z)$ do I plug into the equation? Is it simply the average of the topic proportion associated with topic $z$ across all the training documents, i.e., $P(z) = \sum_{i=1}^{d}\frac{P(z_i)}{d}$ where $d$ is the number of training documents and $P(z_i)$ is the topic proportion for topic $z$ in document $i$?

I've also looked into here and here, but none seems to address my question. Thank you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.