$\newcommand{\Dirichlet}{\operatorname{Dirichlet}}$In a Dirichlet Process, where $P \sim DP(\alpha P_0)$ and $y_i \sim P$ are iid, the posterior distribution of a Dirichlet Process $P$ is usually given as:

$$ P \mid y_1, \ldots y_n \sim DP\left(\alpha P_0 + \sum_{i}^{n}\delta_{y_i}\right) $$ where $\delta_{y_i}(x)$ is the Dirac delta mass function taking on value 1 if $x=y_i$.

It is also equivalent that for any measurable partition $B_1, \ldots, B_k$:

$$ P(B_1), \ldots, P(B_k) \mid y_1, \ldots y_n \sim \Dirichlet\left(\alpha P_0(B_1)+\sum_{i}^{n}\mathbb{1}_{y_i \in B_1}, \ldots, \alpha P_0(B_k)+\sum_{i}^{n}\mathbb{1}_{y_i \in B_k}\right) $$

In the limiting case, for $\alpha \to 0$, we have that:

$$ P(B_1), \ldots, P(B_k) \mid y_1, \ldots y_n \sim \Dirichlet\left(\sum_{i}^{n}\mathbb{1}_{y_i \in B_1}, \ldots, \sum_{i}^{n}\mathbb{1}_{y_i \in B_k}\right) $$

Here, suppose I had some data and I wanted to plot realizations from the posterior of the limiting case above. In order to do so, I must specify bin size before I can sort the data into bins.

Question: Is there any general guideline for specifying the bin size?

My understanding of the Dirichlet Process is that it sort of is independent of having to specify a bin size, so I am very confused about how to do this. In general, what I seek to do is:

1) First specify number of bins $k$.
2) Then sort the data into the bins.
3) Then each bin will have a corresponding count, i.e., 4 data points in first bin, 2 in second, etc.
4) Put the counts into the Dirichlet above to get realizations.

Do I fix $k$ here? thanks.

  • 3
    $\begingroup$ What bins are you talking about? Do you mean what partition to use? $\endgroup$ – jaradniemi Dec 6 '15 at 2:27

In the limiting case, when $\alpha \rightarrow 0$, all samples from $P$ will have the same value. This can easily be illustrated if we look at the predictive distributions of $y_1$ and $y_2$: $$ \begin{eqnarray*} y_1 &\sim& P_0 \\ y_2|y_1 &\sim& \frac{\alpha}{\alpha+1} P_0 + \frac{1}{\alpha+1}\delta_{y_1} \end{eqnarray*} $$

Except for the first sample, $y_1$, all others will have zero probability of getting a new value drawn from the base distribution, $P_0$, and will almost surely form a single cluster.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.