I am puzzled by how this Gibbs sampler on section 6 of Escobar & West (1995) works. To put it in simple words, the aim is to sample $\alpha$. The defined terms are: $$\eta\sim \texttt{Beta}(a,b)$$ and $$\alpha \sim \pi \texttt{Gamma}(\theta,f(\eta))+(1-\pi)\texttt{Gamma}(\theta-1,f(\eta))$$ the paper says (with a bit of simplification)

It is now clear how $\alpha$ can be sampled at each stage of the simulation. At each Gibbs iteration, we first sample $\eta$ from the defined Beta distribution, and use the sampled $\eta$ and the fixed $\theta$ to sample $\alpha$ from the mixture of the Gamma distributions.

the confusing bit is,

On completion of the simulation $p(\alpha|\texttt{Data})$ will be estimated by the usual Monte Carlo averaging $p(\alpha|\texttt{Data})=\sum_{s=1}^{N}p(\alpha|\theta,\eta_s)$, where $\eta_s$ are the sampled values of $\eta$.

Knowing that the aim in here was to sample $\alpha$, why do we need to estimate $p(\alpha|\texttt{Data})$? We already have a sample for $\alpha$, so what is the need to estimate its probability. Also not sure why can we plug in all the sampled values of $\eta$ in this estimation, shouldn't one just use the sampled $\eta$ based on which we sampled the corresponding $\alpha$?

My only explanation: Given all the sampled $\alpha$ (let's put them in a set $S$) for each sampled $\alpha$, we need to compute it's posterior $P(\alpha|\texttt{Data})$. For this, we use all the sampled values for $\eta$ from all the Gibbs iterations to compute the summation. This way each sampled $\alpha$ will get a Monte Carlo averaged posterior estimate. Using the accumulation of all these posterior estimates based on which we sample an $\alpha$ using accumulated posterior estimates of all sampled $\alpha$ in $S$. Is this the correct explanation?

Escobar, M. D., & West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the american statistical association, 90(430), 577-588.


The paper is about Bayesian estimation and $\eta$ is a prior. Given your data and the priors you can estimate posterior probabilities. Posterior probabilities are calculated because the paper is about density estimation, so you use their method since you are interested in the density itself. If you were interested in something else, you could use the MCMC samples to estimate any quantities of interest, as you correctly noticed.

  • $\begingroup$ right, but I am still puzzled with a key concept: to have a proper sampler for $\alpha$ do we need to take i.e. 100 sampled values of $\alpha$ and do the monte carlo averaging (as I described in the last paragraph of my post)? Or any of those sampled $\alpha$ will be good enough? $\endgroup$ – user3639557 Aug 2 '16 at 13:09
  • $\begingroup$ @user3639557 I'm not sure if I understand you correctly, but the main idea is: you draw large number of samples and then compute the quantities of interest from those samples treating them as your estimates. This is how MCMC works. $\endgroup$ – Tim Aug 2 '16 at 14:18
  • $\begingroup$ forget what I ask. Can you verify how they "sample" $\alpha$? $\endgroup$ – user3639557 Aug 3 '16 at 5:38
  • $\begingroup$ @user3639557 unless you make bugs in your code there are theorems that show that with large enough sample it's going to converge. And if you ask how to check if your model fits the data then goggle "posterior predictive checks". $\endgroup$ – Tim Aug 3 '16 at 5:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.