I am trying to design a collapsed Gibbs sampler on a mixture model based on Hierarchical Dirichlet Process ($g\sim DP(\gamma, b)$, $\pi\sim DP(\alpha, g)$ ). Should I resample from the posterior of the parent DP at the end of each loop over the dataset? Can I sample only once from the parent DP and do multiple loops over the dataset?

If I have done the derivation correctly a collapsed Gibbs sampler for this would be

$ p(h_i=k|h^{-i}, d, \pi, g, b, \theta, \xi) \propto p(h_i=k|h^{-i}, g, b) p(d^i|d^{-i}, h_i=k, h^{-i}, \xi) $

enter image description here

  • $\begingroup$ We need more information about the context, the variables in the posterior, and the exact problem setup to answer this question. $\endgroup$ – Greenparker Oct 22 '17 at 9:47
  • $\begingroup$ Thanks for your reply. I attached my graphical model above. I am wondering whether I should resample from the DP $g$ over the iterations of the Gibbs sampler over my dataset. As far as I understand, this would imply reinitiliazing the latent variables $h$ at each such iteration and I am unsure how this affect the burn-in of the sampled Markov chain $\endgroup$ – Dionysis M Oct 23 '17 at 10:51
  • $\begingroup$ I'm considering that it suffices to update the distribution of $g$ after every pass over the dataset and accordingly adapt the posterior of $ h $. But initialization of $ h $ should take place only once, in the beginning of the MCMC. $\endgroup$ – Dionysis M Oct 24 '17 at 16:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.