Mean of draws of Dirichlet Process

Question

Lets assume a Dirichlet process random measure in stick-breaking notation $G=\sum^\infty_{i=1} p_i \delta_{\lambda_i}$, such that $\lambda_i\sim H$ from some base distribution H, with point mass $\delta$ at $\lambda_i$ and $p_i$ contructed by Beta(1,$\alpha$) stickbreaking. Now a draw of this process is a discrete probability measure.

When I have N samples from a truncated version of this Dirichlet process, either directly from the prior or from some posterior distribution (which is a DP again), what is the notion of a "mean cdf" derived from all samples? And how can one understand credible sets in the case of a posterior that is basically a set of cdfs?

Answer 1

Consider a set $S$ which can be partitioned to $B_{1}\cup B_{2} =S$ and $B_{1}\cap B_{2}=\varnothing$. A Dirichlet Process can be considered as a prior over distributions. So, imagine that you have the distribution $G$ that you want to model and learn about, the Dirichlet process will place a prior on that $G$, i.e.

$$(G(B_{1}),G(B_{2}))\sim DP(H,a)$$

where $\mathbb{E}[G(B_{1})]=H(B_{1})$, where $H$ is the mean of the Dirichlet process. For example, you could say that the distribution $G$ that you are interested in is centered around $H=Normal(0,1)$.

Now in the case were you have obsrveded samples $X_{1},X_{2},...,X_{n}$ you update your prior $DP(H,a)$, to

$$DP(\frac{a}{a+n}H+\frac{1}{a+n}\sum_{i=1}^{n}\delta_{x_{i}},a+n)$$

Now the posterior mean over the distribution at $G(B_{1})$ will be

$$\mathbb{E}[G(B_{1})] = \frac{\frac{a}{a+n}H(B_{1})+\frac{1}{a+n}\sum_{i=1}^{n}\delta_{x_{i}}(B_{1})}{\frac{a}{a+n}H(B_{1})+\frac{1}{a+n}\sum_{i=1}^{n}\delta_{x_{i}}(B_{1}) + \frac{a}{a+n}H(B_{2})+\frac{1}{a+n}\sum_{i=1}^{n}\delta_{x_{i}}(B_{2})} \\ =\frac{a}{a+n}H(B_{1})+\frac{1}{a+n}\sum_{i=1}^{n}\delta_{x_{i}}(B_{1}) $$

In a similar way you can calculate the rest.

A really good book that you can read is the "Bayesian Nonparametric Data Analysis" of Peter Müller.