1
$\begingroup$

enter image description hereI have been having difficulty understanding why Algorithm 8 of Neal (2000) is a valid sampler.

I am looking for lecture notes that include a nice explanation of the proof. Does anyone know of any such reference?

$\endgroup$
3
  • $\begingroup$ Please provide the background so that a reader does not need reading Neal (200) first. $\endgroup$
    – Xi'an
    Commented Oct 30, 2019 at 8:02
  • $\begingroup$ Thank you for the edit, I will be sure to do this in the future. $\endgroup$
    – Tomislav
    Commented Oct 30, 2019 at 16:03
  • $\begingroup$ @Xi'an It's a good reading though :D $\endgroup$
    – fm361
    Commented Nov 2, 2022 at 8:51

1 Answer 1

2
$\begingroup$

For brevity, I'm omitting the normalizing constants. Under the standard (Polya urn) Gibbs Sampler for a vanilla DP, the model would be:

$$ P(c_i = c\mid \ldots) \propto \begin{cases} n_{\neg i, c}f(Y_i\mid\theta_c) &\text{ for }1 \leq c \leq k^{\neg i}\\ \alpha\int_{\theta} f(y_i\mid\theta)dG_0(\theta) &\text{ for }c = k^{\neg i} + 1 \end{cases} $$

Doing that integral analytically is difficult, if $G_0$ is not conjugate to $F$.
Some of Neal's algorithms offer various perturbations of this model trying to bypass the need for conjugacy, by doing that integral implicitly. By sampling $\theta$ from $G_0$, then weighting by $f(y_i\mid\theta)$, you effectively get samples from $f(y_i\mid\theta)dG_0(\theta)$. In fact, I think algorithm 7 does just this (with a single sample per iteration). Algorithm 8 recognizes that a single sample doesn't represent that integral very well, so we generate $m$ samples from $G_0$, and weight them accordingly.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.