1
$\begingroup$

Say I want to run topic modeling with LDA on The 20 newsgroups text dataset. So basically a dataset with texts where every text belongs to one of 20 categories. I want the LDA to split the documents into 20 clusters matching the original document categories.

I can think of 2 approaches, but I'm not sure which one is correct. (Skipping documents preprocessing int the code examples below for brevity)

I'm also aware that the dataset has targets and supervised learning methods could be used.

Approach 1

We have 20 categories, so we need LDA to learn 20 topics. So the code would look like:

LDA.Learn(topics=20, dataset)

results=[]
for doc in documents:
    topics = LDA.Predict(doc) // topics is a vector of 20 probabilities
    topic = argmax(topics) // we take the most likely topic
    results.append(topic)

Approach 2

Let's make LDA learn an arbitrary number of some abstract topics, say 100. Then cluster the outputs into 20 categories. So the code would look like:

LDA.Learn(topics=100, dataset)

outputs=[]
for doc in documents:
    output = LDA.Predict(doc) // output is a vector of 100 probabilities of the document belonging to every of the 100 abstract topics
    results.append(outputs)

results = SomeClusteringAlg(k=20, outputs)

Questions

  1. Which of the approaches above is correct and why?
  2. Can we expect LDA topics to map directly to the wanted categories? (approach 1)
  3. Or should we rather think of it as a dimensionality reduction method outputting some abstract representation? (approach 2)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.