4
$\begingroup$

I am trying to understand and implement the blocked Gibbs sampler described on page 552 in Bayesian Data Analysis by Gelman et al. in the context of using a Dirichlet process as a prior in a mixture model. The three steps are as follows:

  1. Update $S_i \in \{1, \dots, N\}$ (cluster allocations) by multinomial sampling.
  2. Update the stick-breaking weight $V_c$, $c = 1, \dots, N - 1$, from a suitable beta.
  3. Update $\theta_c^*$, $c = 1, \dots, N$ (cluster parameters), exactly as in the finite mixture model, with the parameters for unoccupied clusters with $n_c = 0$ sampled from the prior $P_0$.

The first two steps are more or less clear. The third one, however, is not so much, and I would appreciate if someone could explain the process. How exactly are the parameters updated?

For more context, if needed, I assume a Gaussian distribution for the data with an unknown mean and an unknown precision (hence, $\theta_i^* = (\mu_i, \tau_i)$) and use a Gaussian–gamma distribution as a conjugate prior (which is $P_0$).

$\endgroup$
2
  • $\begingroup$ Does this article help? There is a section about collapsed Gibbs sampling which is another term for blocked Gibbs sampling. You can also check Yee Whye Teh's implementation of DP here. $\endgroup$ – kedarps Sep 3 '20 at 20:07
  • $\begingroup$ Thank you. I am specifically interested in the one described in the book on page 552. $\endgroup$ – Ivan Sep 6 '20 at 9:11
0
$\begingroup$

I have figured it out and written a blog post explaining the whole procedure, including an implementation in R. For those interested, please find it here:

https://blog.ivanukhov.com/2021/01/25/dirichlet-process.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.