1
$\begingroup$

I have a (hopefully simple) question about notation when defining a DP. I have read a lot of papers on DPs, but this is not clear to me, or at least I have not noticed a convention.

Say that I am sampling from a simple Gaussian mixture DP, defined as the following generative model:

\begin{align} x~|~\mu_i&\sim N(\mu_i, 1)\\ \mu_i~|~G&\sim G\\ G&\sim\text{DP}(\alpha,G_0) \end{align}

$G_0$ will also be associated with a generative distribution, in this case let's assume $N(0, 10)$, but obviously this can be arbitrarily complex depending on the application, perhaps with it's own hyper-parameters.

If I'm performing inference, say using Neal's (2000) Algorithm 8, then I use samples from $G_0$, in this case $N(0, 10)$, for the potential new classes.

My question therefore: is there a standard (and succinct) way to define the distribution of $G_0$? Ideally I feel that it would be sensible and logical to define it as part of the generative model so that the whole model definition is in one place, but writing something like $G_0\sim N(0,10)$ isn't quite accurate, and doesn't naturally extend to more complex models, say where $G_0$ contains samples of both unknown mean and variance.

$\endgroup$

1 Answer 1

1
$\begingroup$

I'm going to post an answer here as a proposal, adding a second parameter to the base distribution.

I would define the model as follows:


\begin{align} x~|~\mu_i&\sim N(\mu_i, \sigma^2_i)\\ \mu_i,\sigma^2_i~|~G&\sim G\\ G&\sim\text{DP}(\alpha,G_0)\\ \end{align} where $G_0$ is a distribution over $\left\{\mu,\sigma^2\right\}$ such that \begin{align} G_0^\mu&= N(0, 10)\\ G_0^{\sigma^2}&= \text{Inv-Gamma}(3, 1) \end{align}


I think that this is fairly clear, but I would be interested to hear alternatives or comments.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.