1
$\begingroup$

I'm going over the paper by Hoffman et. al. titled "Stochastic Variational Inference" (SVI) where the authors present an algorithm for Hierarchical Dirichlet Process (HDP) topic models. I'm comparing this with a paper by Chong Wang et. al. titled "Online Variational Inference for the Hierarchical Dirichlet Process" where the authors present an online HDP algorithm (also implemented in gensim). And I'm wondering between the differences of these two algorithms. In what ways SVI HDP is different from online HDP?

$\endgroup$
1
$\begingroup$

There is not a difference; the online method in Chong et al. is an instance of the method later generalized in Hoffman et al., a fact that is tucked into a footnote in the SVI paper:

This section is organized as follows. We first give some background on the Dirichlet process and its definition via Sethuraman’s stick breaking construction, which is a distribution on the infinite simplex. We then show how to use this construction to form the HDP topic model and how to use stochastic variational inference to approximate the posterior.$^7$

  1. This algorithm first appeared in Wang et al. (2011). Here we place it in the more general context of Section 2 and relate it to stochastic inference for LDA.
$\endgroup$
  • $\begingroup$ interesting. nice catch! $\endgroup$ – Vadim Smolyakov Jun 20 '17 at 18:10
  • $\begingroup$ @VadimSmolyakov Read a paper as many times as I have the SVI paper, you remember a few things :) $\endgroup$ – Sean Easter Jun 24 '17 at 1:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.