0
$\begingroup$

In the formal definition of Dirichlet process, we know that $\sum_{k=1}^nX(B_k)=1$ by the property of Dirichlet distribution. My question is, is it true that $\sum_{k=1}^nH(B_k)=1$? Why?

$\endgroup$
1
$\begingroup$

The statement $\sum_{k=1}^n H(B_k) = 1$ is true because the base distribution $H$ is a probability distribution and so its sum over all realizations must be equal to one. Since this isn't a very satisfying answer, let's look at the definition of the DP from that Wikipedia page. The formal definition is:

$(X(B_1),...,X(B_n)) \sim Dir((\alpha H(B_1),...,\alpha H(B_n))$

Now, if we look at another definition of the Dirichlet distribution referenced in the $Dir$ above at https://en.wikipedia.org/wiki/Dirichlet_distribution , we see that the function's argument is a vector of positive values $ \alpha_1,...\alpha_k$. We can sum that up into a scalar concentration parameter $\alpha$ and the vector of probabilities $H$ such that $\alpha\sum_k^K(H_k) = \sum_k^K \alpha_kJ_k$, where $J$ is some other probability distribution which is being modified by those $\alpha$ coefficients Sorry for double dipping on $\alpha$ here; the two wikipedia pages have two slightly different meanings of that symbol (vector versus scalar).

One way to understand the Dirichlet process is the 'stick breaking' construction. I encourage you to read about that here: https://en.wikipedia.org/wiki/Dirichlet_process#The_stick-breaking_process . In this representation, if the sum of $H$ exceeds 1, then the DP would be trying to 'break off' parts of the stick which we didn't have to begin with.

In short, it is true because we defined $H$ to be a base probability distribution which is then rescaled using the concentration parameter which can be either a scalar for a symmetric Dirichlet distribution or a vector for a nonsymmetric one.

$\endgroup$
1
  • $\begingroup$ Thanks for your mention of the stick breaking process. In the section, since $\theta_k$s are sampled from a distribution, is it possible that for $j\neq i$, $\theta_i=\theta_j$? If the base distribution is continuous, it's "almost surely impossible". However, if it's discrete or has finite support, then it's more likely that $\theta_i=\theta_j$. How is it dealt with in Dirichlet process? (not directly related to the original question, but derived from the answer, and asked in case you know the answer). $\endgroup$ Jul 14 '15 at 0:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.