In a Dirichlet process, can the base distribution be discrete?

Question

Must it be continuous? Note we are talking about the base distribution. The sampled distribution is discrete.

1) If the base distribution is continuous, drawing from it will get a new value (a new cluster mean, for example) every time.

2) But, you can nest Dirichlet processes. Which implies the base distribution can be discrete. (You can place a prior over your base distribution which is just another Dirichlet process).

Consider the Blackwell-Macqueen urn scheme.

H is a distribution over colors.

Start with an empty urn.

With probability alpha sample a color from H. Note, if H is continuous, you'll get a different color every time. If the H is discrete, you may draw colors already present in the urn.

With probability proportional to n-1, pick a ball from the urn. Put the ball back and another one of the same color.

Reference: http://www.columbia.edu/~jwp2128/Papers/PaisleyWangetal2015.pdf

edit: The paper above demonstrates this is possible. Any distribution drawn for the a DP with a discrete base distribution will have the same atoms as the base distribution, but with different weights.

Still, I wonder how the CRP would work under this scheme since there's a chance you'd select a non-empty table (analogously to the above urn scheme).

Answer 1

The base measure can be discrete. An example of the case where it is discrete is the Hierarchichal Dirichlet Process, wherein $F_j \sim \mathcal D(\alpha F_0)$ where $F_0 \sim \mathcal D(\gamma H)$. The base measure of the second level of the hierarchy ($F_0$) is discrete with probability $1$.