Latent Dirichlet Allocation - basic question about generative process

Question

After trying to understand Blei's 2009 paper "Topic Models" and reading several websites. This blog entry is the simplest explanation I could find. I still don't understand how LDA works.

According to the blog entry mentioned previously LDA consists of two steps:

• The generative process
• The learning (when a new set of documents arrives and we wish to discover its topical compositions)

The blog mentions that in this learning process we start with random assignments and then we compute: p(topic t | document d) and p(word w | topic ) and reassign the word to a new topic, where we choose topic t with probability p(topic t | document d) * p(word w | topic t) .

I don't see where in this learning process we use the "output" of the generative process and what exactly is this "output".

Answer 1

When we call LDA a generative model, we mean that it assumes a generative process for the data. Namely, a topic mixture is sampled for each document, a topic for each word, and a word from that topic. From this perspective, the observed data is the output.

From LDA's assumed model, we can use Bayes' rule to infer latent topics and documents (topic mixtures). Though the blog demonstrates collapsed Gibbs sampling, you can perform this in a few ways. (For more, see slide 21 from the deck associated with this lecture).

However you perform this inference, you use the training corpus as the observed data in Bayes' rule. In the Gibbs example you've written above, each word is output of the generative process.