I am trying to recreate the model proposed by Gao et al. (2011), based on the Hierarchical Dirichlet Process proposed by Teh and al. (2005).

To estimate the model (let's call it iHDP) I need to implement a Gibbs sampler as described in Section 5.3 in Teh et al. (2005). I think I understand all the elements of this process, apart from estimating the probability of an assignment to a new cluster - Eq. 37 in Teh and al. (2005), repeated in Eq. 9 in Gao et al. (2011).

By analogy to Gibbs sampler for LDA I think I know how to compute $f_k^{-x_{ij}}(x_{ij})$ - from a posterior of the $k$-th topic, using its term counts. But how do I calculate $f_{k_{new}}^{-x_{ij}}(x_{ij})$? Teh and al. (2005) write:

$f^{−x_{ji}}_{k_{new}}(xji) = \int f(x_{ji}|\phi)h(\phi)d\phi$ is simply the prior density of $x_{ji}$.

while Gao et al. (2011) write:

$f^{−x_{ji}}_{k_{new}}(x_{ji}) = p(x_{ij})$

It seems to me that this is just the probability of getting whatever the term in $x_{ij}$ is. Suppose I'm using a symmetric Dirichlet distribution as a base distribution for the topics, would that mean that mean that $f^{−x_{ji}}_{k_{new}}(x_{ji})$ is just $\frac{1}{|V|}$ where $|V|$ is the size of the vocabulary? It doesn't seem very intuitive, especially because the superscript $-x_{ij}$ is said to mean "considering all other observations".

Could someone perhaps offer some intuition behind this, or correct my reasoning if it's wrong?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.