I recently read paper by Chong Wang and David M. Blei "Variational Inference for the Nested Chinese Restaurant Process". And I couldn't understand the next part (from p.5):

The variational update functions for W and x depend on the actual distributions we use, and deriving them is straightforward. If they include an infinite sum then we apply similar techniques as we did for q(vi).

Could anyone explain (and give an example) how to implement variational update for W and x?

And if there are any other good papers/tutorials/code to understand variational inference for hLDA?

Thanks for any suggestions.

  • 6
    $\begingroup$ Well, it's not just you. I feel like there are 5 guys writing all the Bayesian nonparametrics papers. They all write the same way, and lose me in exactly the same spot in every paper. Unfortunately, that spot is before page 5. $\endgroup$
    – Ben Ogorek
    Apr 14 '14 at 14:54

David Blei has an implementation of hLDA on his website, though I'm not sure if it's variational or MCMC. It's the second one from the bottom in the software list.

  • $\begingroup$ this implementation uses gibbs sampling. Such a feeling, that all available implementations of hLDA relied on MCMC sampling. But I'm interested in variational inference, because this method is more scalable. $\endgroup$
    – peppered
    Dec 12 '12 at 21:19
  • $\begingroup$ @pertsevin, if it's a pure Gibbs implementation, shouldn't it scale pretty well? $\endgroup$ Nov 8 '13 at 14:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.