Dirichlet process mixture MCMC

Question

I'm reading Markov Chain Sampling Methods for Dirichlet Process Mixture Models by Radford M. Neal. Equation (3.6) states that

$$ \text{If } c=c_{j} \text{ for some } j\neq i: P\left(c_{i}=c\;|\;c_{-i}, y_{i}, \boldsymbol{\phi}\right) = b\frac{n_{-i,c}}{n-1+\alpha}F\left(y_{i},\phi_{c}\right)$$

$$P\left(c_{i}\neq c_{j} \text{ for all } j\neq i\;|\; c_{-i},y_{i},\boldsymbol{\phi}\right)=b\frac{\alpha}{n-1+\alpha}\int F\left(y_{i},\phi\right)\,dG_{0}\left(\phi\right)$$

It doesn't give any derivation. How can I compute this by hand?

Answer 1

$$ P(c_i=c|\vec{c_{-i}},y_i,\phi) =\frac{p(\vec{c_i},y_i,\phi)}{p(\vec{c_{-i}},y_i,\phi)} = \frac{p(y_i|\vec{c_{i}},\phi)p(\vec{c_{i}},\phi)}{p(y_i|\vec{c_{-i}},\phi)p(\vec{c_{-i}},\phi)} \\ = \frac{p(y_i|c_i,\phi)p(c_i|\vec{c_{-i},\phi})p(\vec{c_{-i}},\phi)}{p(y_i|\vec{c_{-i}},\phi)p(\vec{c_{-i}},\phi)} \\= \frac{p(y_i|c_i,\phi)p(c_i|\vec{c_{-i}},\phi)}{p(y_i)} $$

Here, $p(c_i|\vec{c_{-i}},\phi)=\frac{n_{-i,c}}{n-1+\alpha}$.

When $c_i$ is a existing one, then:

$$ p(y_i|c_i,\phi)=F(y_i,\phi_c) $$

When $c_i$ is a new cluster, then:

$$ p(y_i|c_i,\phi)=\int p(y_i|\phi_c,c_i)p(\phi_c|\phi,c_i)d\phi_c $$ Since $dG_0=p(\phi_c|\phi)$, we can conclude that: $$ p(y_i|c_i,\phi)=\int F(y_i,\phi_c)dG_0 $$

This is what I thought about the derivation. I am not quite understand that if it is correct. Did you solve this problem? @Daeyoung Lim