28
$\begingroup$

It seems to me that only two R packages are able to perform Latent Dirichlet Allocation:

One is lda, authored by Jonathan Chang; and the other is topicmodels authored by Bettina Grün and Kurt Hornik.

What are the differences between these two packages, in terms of performance, implementation details and extensibility?

$\endgroup$

closed as off-topic by gung May 10 '18 at 13:53

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question appears to be off-topic because EITHER it is not about statistics, machine learning, data analysis, data mining, or data visualization, OR it focuses on programming, debugging, or performing routine operations within a statistical computing platform. If the latter, you could try the support links we maintain." – gung
If this question can be reworded to fit the rules in the help center, please edit the question.

26
$\begingroup$

Implementation: The topicmodels package provides an interface to the GSL C and C++ code for topic models by Blei et al. and Phan et al. For the earlier it uses Variational EM, for the latter Gibbs Sampling. See http://www.jstatsoft.org/v40/i13/paper . The package works well with the utilities from the tm package.

The lda package uses a collapsed Gibbs Sampler for a number of models similar to those from the GSL library. However, it has been implemented by the package authors itself, not by Blei et al. This implementation therefore differs in general from the estimation technique proposed in the original papers introducing these model variants, where the VEM algorithm is usually applied. On the other hand, the package offers more functionality then the other package. The package provides text mining functionality too.

Extensibility: Regarding extensibility, the topicmodel code by its very nature can be extended to interface other topic model code written in C and C++. The lda package seems to be more relying on the specific implementation provided by the authors, but there Gibbs sampler might allow specifying your own topic model. For extensibility issues nota bene, the former is licensed under GPL-2 and the latter LGPL, so it might depend on what you need to extend it for (GPL-2 is stricter regarding the open source aspect, i.e. you can't use it in proprietary software).

Performance: I can't help you here, I only used topicmodels so far.

Conclusion:
Personally I use topicmodels, as it is well documented (see the JSS paper above) and I trust the authors (Grün also implemeted flexmix and Hornik is R core member).

$\endgroup$
  • 7
    $\begingroup$ First, I'm sure that topicmodels is an excellent package, and I agree that it is very well documented. Regarding "trusting" the authors of the LDA package, Jonathan Chang was my PhD student, and I completely trust his code. He is both an excellent software engineer and scholar. His package is easy, scales well to fairly large collections, and it is nice to explore the results in R. (I like and I'm used to the sparse matrix representation of documents.) Of note, LDA implements other models, including relational topic models, supervised LDA (with GLM), and mixed-membership stochastic blockmodel. $\endgroup$ – user54831 Aug 28 '14 at 2:26
  • $\begingroup$ Thanks for weighing in. I'm sure the lda package is great, I didn't mean to imply that lda has somehow inferior code. I stated my personal impression (the documentation appeared a bit sloppy). Since that was >2 years ago I edited the answer slightly (the typos at the CRAN page are still there, I think it would be good for them to be fixed but that email seems to have gotten lost). $\endgroup$ – Momo Aug 28 '14 at 12:07
  • $\begingroup$ I guess you have a lot of experience with both packages and know most about the general topic, how about providing an answer listing some of the pros and cons of both packages? I didn't use lda, so I couldn't give an empirical assessment of it. Providing that would surely help the OP and everyone else as well as correct any possible injustices (which were not intended). Thanks! $\endgroup$ – Momo Aug 28 '14 at 12:12
  • 1
    $\begingroup$ I also tried both, and found the lda package to contain more options. However, it's difficult to understand how to format your data in order for it to fit into the model. The topicmodels package works well with the tm package, while the lda package requests a list which is not clear how to create. $\endgroup$ – Omri374 Dec 29 '15 at 11:39
12
$\begingroup$

+1 for topicmodels. @Momo's answer is very comprehensive. I'd just add that topicmodels takes input as document term matrices, which are easily made with the tm package or using python. The lda package uses a more esoteric form of input (based on Blei's LDA-C) and I've had no luck using the built-in functions to convert dtm into the lda package format (the lda documentation is very poor, as Momo notes).

I have some code that starts with raw text, pre-processes it in tm and puts it through topicmodels (including finding the optimum number of topics in advance and working with the output) here. Could be useful to someone coming to topicmodels for the first time.

$\endgroup$
  • 3
    $\begingroup$ For those not already using tm, and wanting to play with lda, JFreq also puts plain texts in LDA-C's preferred format. $\endgroup$ – conjugateprior Mar 11 '12 at 15:45
  • $\begingroup$ Good to know about JFreq, I haven't seen it before. Thanks for the tip! $\endgroup$ – Ben Mar 13 '12 at 1:08
  • 1
    $\begingroup$ I just spotted the read_dtm_Blei_et_al function in the tm package which does the same thing. $\endgroup$ – Ben Mar 23 '12 at 7:26
3
$\begingroup$

The R Structural Topic Model (STM) package by Molly Roberts, Brandon Stewart and Dustin Tingley is also a great choice. Built on top of the tm package it's a general framework for topic modeling with document-level covariate information.

http://structuraltopicmodel.com/

The STM package includes a series of methods (grid search) and measures (semantic coherence, residuals and exclusivity) to determine the number of topics. Setting the number of topics to 0 will also let the model determine an optimum number of topics.

The stmBrowser package is a great data visualization complement to visualize the influence of external variables on topics. See this example related to the 2016 presidential debates: http://alexperrier.github.io/stm-visualization/index.html.

$\endgroup$
0
$\begingroup$

I used all three libraries, among all 3 viz., topicmodels, lda, stm; not everyone works with n grams. The topicmodels library is good with its estimation and it also work with n grams. But if anyone is working with uni grams then the practitioner may preferred stm as it gives structured output.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.