0
$\begingroup$

I've been working with LDA (Latent Dirichlet Allocation topic model) for a while now and I believe I have an intermediate understanding of it.

The unsupervised nature of LDA is one of its big advantages, but can also be a drawback - particularly when you want to estimate "finer/smaller" topics. There are several models that try to include additional "lexical priors" (i.e. "I believe those two words should be in the same topic etc.), e.g. GuidedLDA or Dirichlet Forest.

It seems to me, however, that the most intuitive way of doing this would be to directly modify the topic priors I usually assume that all topics came from the same root distribution: $$\phi_k \sim \text{Dir}(\eta), \quad \forall k$$ However, if I believe that a topic would heavily feature some set of words I can just increase those words' prior probability for a single topic and voila, right? So generally, for one "custom" topic: $$\phi_1 \sim \text{Dir}(\eta^*)$$ $$\phi_k \sim \text{Dir}(\eta), \quad \text{for } k\neq 1$$

The problem is that I cannot find any information about such approach, no guidelines, no research using it. So my general question is - how come? Am I missing something here? Or am I just bad at looking for it, maybe it has some particular name?

Improve this question
$\endgroup$
2
  • $\begingroup$ How would you know before running inference that one topic heavily features some set of words? $\endgroup$
    – kedarps
    Commented Jun 17, 2019 at 17:06
  • $\begingroup$ By some "prior lexical knowledge" - I'm using it to get more detailed topics when I know (or want) some words to appear together. For example, it's the motivation behind GuidedLDA/SeededLDA as used here, it's just that they end up using a much more complicated approach and I don't really see why. $\endgroup$
    – yassem
    Commented Jun 17, 2019 at 18:05

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.