3
$\begingroup$

I am reading the paper "Bayesian Analysis of Some Nonparametric Problems" by Ferguson where the Dirichlet process is introduced. There is a proposition 5 where the joint distribution of Random Variables(RV) sampled from a Dirichlet distribution is given. I am able to understand most of the proof but having trouble in the last step. I will try to reproduce all the required details here.


Proposition 5: Let $P$ be a Dirichlet process on $(\mathscr{X}, \mathscr{A})$ with parameter $\alpha,$ and let $X$ be a sample of size 1 from $P .$ Let $\left(B_{1}, \cdots, B_{k}\right)$ be a measurable partition of $\mathscr{X}$, and let $A \in \mathscr{A}$. Then, \begin{aligned} \mathscr{P}\left\{X \in A, P\left(B_{1}\right)\right.&\left.\leqq y_{1}, \cdots, P\left(B_{k}\right) \leqq y_{k}\right\} \\ &=\sum_{j=1}^{k} \frac{\alpha\left(B_{j} \cap A\right)}{\alpha(\mathscr{X})} D\left(y_{1}, \cdots, y_{k} \mid \alpha_{1}^{(j)}, \cdots, \alpha_{k}^{(j)}\right) \end{aligned}. Here $D$ is the CDF of the Dirichlet Distribution $\mathscr{D}$ and \begin{aligned} \alpha_{i}^{(j)} &=\alpha\left(B_{i}\right) & \text { if } i \neq j \\ &=\alpha\left(B_{j}\right)+1 & \text { if } i=j . \end{aligned}

Proof: Define $B_{j, 1}=B_{j} \cap A,$ and $B_{i, 0}=B_{i} \cap A^{c}$ for $i=1 \ldots \ldots$. Let $Y_{j, \nu}=P\left(B_{j, \nu}\right)$ for $j=1, \ldots, k$ and $\nu=0,1$. Then, from the definition of a sample from the Dirichlet Process we have $$ \mathscr{P}\left\{X \in A \mid Y_{j, \nu}, j=1, \cdots, k, \text { and } \nu=0,1\right\}=\sum_{\jmath=1}^{k} Y_{j, 1} \quad \text { a.s. } \tag{4}\label{4} $$ Hence for $y_{jv} \in [0,1]$, we can find $$ \mathscr{P}\left\{X \in A, Y_{j, \nu} \leqq y_{j, \nu} \text { for } j=1, \ldots, k, \text { and } \nu=0,1\right\} $$ by integrating the equation $\ref{4}$ with respect to the distribution of the $Y_{j,v}$ over the set $\left\{ Y_{j, \nu} \leqq y_{j, \nu} \text { for } j=1, \ldots, k, \text { and } \nu=0,1\right\}$. This integration is straightforward from the Result $\ref{3}$ and leads to the following result: \begin{equation} \mathscr{P}\left\{X \in A, Y_{j, \nu} \leqq y_{j, \nu} \text { for } j=1, \ldots, k, \text { and } \nu=0,1\right\} = \sum_{j=1}^{k} \frac{\alpha\left(B_{j, 1}\right)}{\alpha(\mathscr{X})} D\left(\mathbf{y} \mid \boldsymbol{\alpha}^{(j)}\right) \tag{5}\label{5} \end{equation}

I was able to follow the proof till here. It is last part of the proof that I find difficult to understand.

The exact statement is the paper is "The conclusion of the proposition follows from this using $\ref{1}$ of the Dirichlet distribution, since $P\left(B_{j}\right)=Y_{j, 0}+Y_{j, 1}$ a.s., and since the process of finding marginal distributions of random variables is linear". I don't understand this statement "..the process of finding marginal distribution of random variables is linear." I cannot quite see how the proposition follows from equation $\ref{5}$ using the result $\ref{1}$. My confusion is that we cannot apply equation $\ref{1}$ because on the RHS of equation $\ref{5}$ we have a mixture of Dirichlet Distribution instead of a Dirichlet Distribution.

Requesting to kindly help me understand the proof. Thanks in advance.


I will state some well known results (of Dirichlet Distribution) here which were used in the proof:

  1. Let $\mathscr{D}$ be the Dirichlet distribution. If $\left(Y_{1}, \cdots, Y_{k}\right) \in \mathscr{D}\left(\alpha_{1}, \cdots, \alpha_{k}\right)$ and $r_{1}, \cdots, r_{l}$ are integers such that $0<r_{1}<\cdots<r_{l}=k,$ then $$ \left(\sum_{1}^{r_{1}} Y_{i}, \sum_{r_{1}+1}^{r_{2}} Y_{i}, \cdots, \sum_{r_{l-1}+1}^{r_{l}} Y_{i}\right) \in \mathscr{D}\left(\sum_{1}^{r_{1}} \alpha_{i}, \sum_{r_{1}+1}^{r_{2}} \alpha_{i}, \cdots, \sum_{r_{l-1}+1}^{r_{l}} \alpha_{i}\right) \tag{1} \label{1} $$

  2. If the prior distribution of $\left(Y_{1}, \cdots, Y_{k}\right)$ is $\mathscr{D}\left(\alpha_{1}, \cdots, \alpha_{k}\right)$ and if $$ \mathscr{P}\left\{X=j \mid Y_{1}, \cdots, Y_{k}\right\}=Y_{j} \quad \text { a.s. } \quad \text { for } j=1, \cdots, k $$, then the posterior distribution of $\left(Y_{1}, \cdots, Y_{k}\right)$ given $X=j$ is $\mathscr{D}\left(\alpha_{1}^{(j)}, \cdots, \alpha_{k}^{(j)}\right),$ where \begin{aligned} \alpha_{i}^{(j)} &=\alpha_{i} & \text { if } \quad i \neq j \\ &=\alpha_{j}+1 & \text { if } i=j \end{aligned} From this formula, we can also derive the following:

  1. Since the following equality holds, \begin{aligned} \mathscr{P}\left\{X=j, Y_{1} \leqq z_{1},\right.&\left.\cdots, Y_{k} \leqq z_{k}\right\} =\mathscr{P}\{X=j\} \mathscr{P}\left\{Y_{1} \leqq z_{1}, \cdots, Y_{k} \leqq z_{k} \mid X=j\right\}, \end{aligned}, we can write the following using the above Result: \begin{array}{l} \int_{0}^{z_{1}} \ldots \int_{0}^{z_{k}} y_{j} d D\left(y_{1}, \ldots, y_{k} \mid \alpha_{1}, \cdots, \alpha_{k}\right) \\ =\frac{\alpha_{j}}{\sum_i \alpha_i} D\left(z_{1}, \cdots, z_{k} \mid \alpha_{1}^{(j)}, \cdots, \alpha_{k}^{(j)}\right) \tag{3}\label{3} \end{array}
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.