2
$\begingroup$

In most of the topic modeling prior literature with LDA, the number of topics is in the range of 50-300.

In big data scenarios, we may need a large number of topics, say 10k or 100k.For example, if we are topic modelling the whole set of Wikipedia articles we need a larger number of topics to better represent the data in topic space.

In such scenarios, what is the best approach for topic modelling where we can get a better representation of the data in topic space, better predictive power with holdout sample?

$\endgroup$
3
  • 1
    $\begingroup$ Better in what sense? Faster? More scaleable? Better predictive power on test data? $\endgroup$ Sep 2 '17 at 14:40
  • $\begingroup$ Better representation of the data in topic space and may be better predictive power on test data! $\endgroup$
    – zapumal
    Sep 2 '17 at 17:04
  • 1
    $\begingroup$ That's helpful. Your question has attracted a few close votes for being unclear, so you may want to edit that in. Saying a bit about what you mean by "better representation" will also help. $\endgroup$ Sep 2 '17 at 17:10
2
$\begingroup$

In terms of held-out prediction, hierarchal Dirichlet process topic models demonstrated performance as well or better than any LDA model in one experiment. (See Fig. 3.) This makes intuitive sense: HDP models are a non-parametric extension of LDA that select the number of topics from the data. We'd expect them to perform as well as the best LDA model. In a big data context, this also saves you the angst of selecting the number of topics ahead of time.

Stochastic variational inference is not a model, but a framework that can train PGMs that meet certain conditions. It's relevant to big data in that it's faster to converge than batch methods, and uses random subsets of the data at each iteration (it needn't hold the full set in memory). The paper introducing SVI uses LDA and HDP examples.

Others models descended from LDA outperform it on held-out data, e.g. Nested Chinese restaurant process models, which find a hierarchy of topics. You might find others in the ~19,900 papers citing LDA as of this writing, with variation in both applicability to big data and performance in particular applications.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.