How does this Sampler work for the Concentration parameter of Dirichlet Process?

Question

I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ and $$\alpha \sim \pi \texttt{Gamma}(\theta,f(\eta))+(1-\pi)\texttt{Gamma}(\theta-1,f(\eta))$$ the paper says (with a bit of simplification)

It is now clear how $\alpha$ can be sampled at each stage of the simulation. At each Gibbs iteration, we first sample $\eta$ from the defined Beta distribution, and use the sampled $\eta$ and the fixed $\theta$ to sample $\alpha$ from the mixture of the Gamma distributions.

the confusing bit is,

On completion of the simulation $p(\alpha|\texttt{Data})$ will be estimated by the usual Monte Carlo averaging $p(\alpha|\texttt{Data})=\sum_{s=1}^{N}p(\alpha|\theta,\eta_s)$, where $\eta_s$ are the sampled values of $\eta$.

Knowing that the aim in here was to sample $\alpha$, why do we need to estimate $p(\alpha|\texttt{Data})$? We already have a sample for $\alpha$, so what is the need to estimate its probability. Also not sure why can we plug in all the sampled values of $\eta$ in this estimation, shouldn't one just use the sampled $\eta$ based on which we sampled the corresponding $\alpha$?

My only explanation: Given all the sampled $\alpha$ (let's put them in a set $S$) for each sampled $\alpha$, we need to compute it's posterior $P(\alpha|\texttt{Data})$. For this, we use all the sampled values for $\eta$ from all the Gibbs iterations to compute the summation. This way each sampled $\alpha$ will get a Monte Carlo averaged posterior estimate. Using the accumulation of all these posterior estimates based on which we sample an $\alpha$ using accumulated posterior estimates of all sampled $\alpha$ in $S$. Is this the correct explanation?

---

Escobar, M. D., & West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the american statistical association, 90(430), 577-588.

Answer 1

The paper is about Bayesian estimation and $\eta$ is a prior. Given your data and the priors you can estimate posterior probabilities. Posterior probabilities are calculated because the paper is about density estimation, so you use their method since you are interested in the density itself. If you were interested in something else, you could use the MCMC samples to estimate any quantities of interest, as you correctly noticed.

Answer 2

Your question can be reformulated as: given MCMC output $\{\eta_s, \alpha_s\}_{s=1}^N$, why not instead use, e.g., a kernel density estimator based on $\alpha_s$? You see this quite often (in computations of marginal likelihoods, log scores, etc) and it is occasionally referred to as “Rao-Blackwellization”. Because the estimator for the density that the paper you refer to uses the conditional distribution rather than the unconditional, its variance is lower. For a simple example, see Rao-Blackwellization of Gibbs Sampler.