6
$\begingroup$

So far I only really understand the Dirichlet Process through its various metaphors. For the Polya Urn scheme, my understanding is that the "base distribution" is the original distribution of colors in the urn before you start the process. So if alpha is very large then somehow your population ends up resembling the base distribution.

But that's as far as my understanding goes. How does the base distribution affect the chinese restaurant process? Or the dirichlet process in general?

$\endgroup$
1
  • $\begingroup$ Keep drawing samples from the DP. Each sample, can be thought of as a multinomial distribution (basically a probability vector). The base is the average of those samples, or to be more precise: the mean of your samples would be close to the base distribution (which itself is another probability vector). Of course, the closeness will be affected by the concentration/strength parameter of DP. $\endgroup$ Jun 7, 2017 at 0:15

1 Answer 1

2
$\begingroup$

Let $$ G \sim \textsf{DP}(\alpha, H) $$ which says that the random distribution $G$ is itself distributed according to the Dirichlet Process with concentration parameter $\alpha$ and base distribution $H$. There is an explicit representation for $G$ and it's useful for understanding the role of the base distribution and its relation to "clustering" that goes on. In particular, $G$ is a discrete distribution with random support points and random weights: $$ G = \sum_{c=1}^\infty w_c\, \delta_{\theta_c} , $$ where $\delta_x$ is a point-mass located at $x$ and $\sum_{c=1}^\infty w_c = 1$. The distributions for the component weights $w = (w_1, w_2, \ldots)$ and corresponding component parameters $\theta = (\theta_1, \theta_2, \ldots)$ are given by \begin{align} w &\sim \textsf{Stick}(\alpha) \\ \theta_c &\stackrel{\text{iid}}{\sim} H . \end{align} The stick-breaking weights are generated according to $$ w_c = v_c \prod_{\ell = 1}^{c-1} (1 - v_\ell) \qquad\text{where $v_c \stackrel{\text{iid}}{\sim} \textsf{Beta}(1,\alpha)$} . $$ The base distribution determines the locations of the support points while the stick-breaking weights determine the amount of clustering.

In the limit as $\alpha \to 0$, the first weight approaches unity: $w_1 \to 1$, in which case the random distribution $G$ has a single support point. In this case, any draw $G$ is quite different from the base distribution $H$. Going the other way, as $\alpha \to \infty$, no finite collection of weights dominate and each random draw of $G$ becomes arbitrarily close to $H$ (i.e., becomes concentrated on $H$).

It is possible to introduce classifications that indicate cluster assignments and integrate out the weights, leaving one with the Chinese Restaurant Process to make the (table) assignments. The base distribution then is used to determine the entrees for the tables.

$\endgroup$
2
  • $\begingroup$ To help clarify the intuition here, could you put this in terms of what a long sample from G would look like under different choices of H? $\endgroup$
    – cgreen
    Feb 27, 2017 at 8:26
  • $\begingroup$ I'm not sure what you're asking for. Let me say two things. First, before G has been drawn, its expectation is H: E[G] = H. Second, after G has been drawn, draws from G don't depend on H; they only depend on the weights and support points of G. $\endgroup$
    – mef
    Feb 27, 2017 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.