In the somewhat inscrutable papers by Blei et al., Dynamic Topic Models and Continuous-time Dynamic Topic Models, there's some ambiguity regarding the role of alpha.

In Dynamic Topic Models, alpha is a logistic normal prior which evolves in discrete time, and yet the variational distribution given in section 3 includes a Dirichlet variational distribution for theta, rather than a normal distribution for eta (as was done in Correlated Topic Models), which indicates that alpha is not logistic-normal but instead Dirichlet.

It could be that this was done in the interests of notational brevity, as the description of the Kalman filter variational inference algorithm focuses specifically on beta, and makes no further mention of alpha.

Continuous-time Dynamic Topic Models is even more vague, making no mention at all of any alpha time-dynamics, the entire paper treats alpha in a fashion identical to LDA.

Does anyone know the preferred construction for (continuous-time) dynamic topic models? Did Blei et al. decide that incorporating time-dynamics into alpha (at the topic proportion level) provided little benefit above and beyond the dynamics of beta (at the topic-word distribution level)?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.