1
$\begingroup$

My specific case is this: There are a number of categories (Science, Health, Religion, etc.), each containing multiple documents. Assume for documents within each category, I perform topic modeling (LDA) with 10 topics ($k=10$). For example, for the Science category, each topic will correspond to Biology, Chemistry, etc.

The topic distribution, $\theta_{category=c}^d$, for each one of the documents, shows how likely it is that they are generated by each of those 10 topic. In this example, $\theta_{category=c}^d$ will be a list of size 10. I want to get the topic distribution over all documents of category $c$, $\theta_{category=c}$. A simple way that I have seen people use is to average each element of $\theta_{category=c}^d$: $$ \theta_{category=c} = mean(\theta_{category=c}^d)$$ But it seems very naive. Is there a better, more rigorous way to combine all these distributions into one?

$\endgroup$
  • $\begingroup$ To clarify, you're looking to find the distribution of topics in each category's corpus? That is, you're not looking to share dependence shared among the categories; is that correct? $\endgroup$ – Sean Easter May 23 '17 at 18:35
  • $\begingroup$ That is correct. I want to get topic distribution per category $\endgroup$ – Alex May 23 '17 at 18:51
1
$\begingroup$

One idea, is that you could concatenate the documents of each category in one big document containing the whole category and infer the topic distribution $\theta_{category=c} $ of this concatenated document. Bear in mind though, that your topic model maybe sensitive to the average length of the initial simple documents. That is, if it's trained on more or less same length documents, then this concatenated document will be $N$ times longer, where $N$ is the number of documents to be concatenated. Maybe, if you use tf-idf features for each document you can alleviate this./

$\endgroup$
1
$\begingroup$

Since you're not introducing any dependencies among the categories, you can continue to treat this as a single LDA model for each category's corpus.

In the notation of the paper introducing LDA, you're looking for $\alpha$, the mean topic parameter for the corpus. I'm not sure which implementation you're using, but R's topicmodels package can estimate this for you. The paper introduces a method for finding an empirical Bayes estimate of $\alpha$ in the appendices, in case you need to roll your own version.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.