4
$\begingroup$

I have an LDA (latent dirichet allocation) model trained over a corpus of documents, where each document is associated with a political party. I'd like to arrive at $p(w|z,party)$ for each word $w$, topic $z$, and party $party$.

From the output of LDA, I have a distribution over topics for each document (i.e. $\theta$) and a distribution over words for each topic (i.e. $\phi$). To combine documents into parties, I'm simply averaging the document-topic distributions for each party's documents. This gives me a distribution over topics for each party, $\theta'$.

First of all, is this a valid thing to do?

Second of all, I now need to get to $p(w|z,party)$ using $\theta'$ and $\phi$. This answer seems to suggest that I can simply do the following:

$p(w|z,party) = \large\frac{\theta'_{party,z}\phi_{z,w}}{\sum_{v \in W}\theta'_{party,z}\phi_{z,v}}$

Is that correct? If so, can someone explain why?

Note: The original paper for LDA uses $\beta$, not $\phi$, for the topic-word distributions. I am using $\phi$ for consistency with the linked question.

Improve this question
$\endgroup$
2
  • $\begingroup$ Have you trained an LDA model for each party, or just one across the entire corpus? $\endgroup$
    – Sean Easter
    Feb 14, 2017 at 13:59
  • $\begingroup$ @SeanEaster Across the whole corpus. $\endgroup$
    – ethan.roday
    Feb 17, 2017 at 17:36

2 Answers 2

Reset to default
3
$\begingroup$

You may want to consider using a structural topic model: http://www.structuraltopicmodel.com/ It's an extension of the correlated topic model (CTM) presented by Blei & Lafferty (2007), allowing for the use of covariates and metadata, such as party in your example. The stm package in R is fantastic, and very easy to use, IMO. There are several published articles showing examples of structural topic models --- see the list of references on the stm website, linked above.

Improve this answer
$\endgroup$
0
$\begingroup$

The Mallet topic modeling package has a DMRTopicModel (explained here: https://mimno.infosci.cornell.edu/papers/dmr-uai.pdf) that lets you condition on a variable. (However, I have tried the DMR model in Mallet and it's extremely difficult to evaluate one model as better than another).

Also, I found this related paper interesting: http://languagelog.ldc.upenn.edu/myl/Monroe.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.