# What does the base distribution of the Dirichlet Process mean?

So far I only really understand the Dirichlet Process through its various metaphors. For the Polya Urn scheme, my understanding is that the "base distribution" is the original distribution of colors in the urn before you start the process. So if alpha is very large then somehow your population ends up resembling the base distribution.

But that's as far as my understanding goes. How does the base distribution affect the chinese restaurant process? Or the dirichlet process in general?

• Keep drawing samples from the DP. Each sample, can be thought of as a multinomial distribution (basically a probability vector). The base is the average of those samples, or to be more precise: the mean of your samples would be close to the base distribution (which itself is another probability vector). Of course, the closeness will be affected by the concentration/strength parameter of DP. Jun 7, 2017 at 0:15

Let $$G \sim \textsf{DP}(\alpha, H)$$ which says that the random distribution $G$ is itself distributed according to the Dirichlet Process with concentration parameter $\alpha$ and base distribution $H$. There is an explicit representation for $G$ and it's useful for understanding the role of the base distribution and its relation to "clustering" that goes on. In particular, $G$ is a discrete distribution with random support points and random weights: $$G = \sum_{c=1}^\infty w_c\, \delta_{\theta_c} ,$$ where $\delta_x$ is a point-mass located at $x$ and $\sum_{c=1}^\infty w_c = 1$. The distributions for the component weights $w = (w_1, w_2, \ldots)$ and corresponding component parameters $\theta = (\theta_1, \theta_2, \ldots)$ are given by \begin{align} w &\sim \textsf{Stick}(\alpha) \\ \theta_c &\stackrel{\text{iid}}{\sim} H . \end{align} The stick-breaking weights are generated according to $$w_c = v_c \prod_{\ell = 1}^{c-1} (1 - v_\ell) \qquad\text{where v_c \stackrel{\text{iid}}{\sim} \textsf{Beta}(1,\alpha)} .$$ The base distribution determines the locations of the support points while the stick-breaking weights determine the amount of clustering.
In the limit as $\alpha \to 0$, the first weight approaches unity: $w_1 \to 1$, in which case the random distribution $G$ has a single support point. In this case, any draw $G$ is quite different from the base distribution $H$. Going the other way, as $\alpha \to \infty$, no finite collection of weights dominate and each random draw of $G$ becomes arbitrarily close to $H$ (i.e., becomes concentrated on $H$).