Timeline for Rao-Blackwellization of Gibbs Sampler
Current License: CC BY-SA 3.0
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 14, 2018 at 2:56 | comment | added | Ding Li | @Greenparker Thanks for pointing out one misunderstanding I have for the result of Gibbs sampling! But still, for the final step here, $\begin{align*} E[\phi \mid data]& =... = \int \phi^* f(\mu \mid data) d\mu. \end{align*}$, we need to supply the marginal distribution of $\mu $, $f(\mu | data)$ right? Why can we use the Gibbs samples directly to do the calculation, as what we get from Gibbs is a joint distribution $f( \mu,\phi | data)$ ? | |
| Apr 12, 2018 at 13:50 | comment | added | Greenparker | @DingLi I don't quite understand your question. When you run the Gibbs sampler, what you obtain are approximate samples from the joint distribution of $(\mu, \phi)$. | |
| Apr 12, 2018 at 2:57 | comment | added | Ding Li | @Greenparker hi, there is one thing that I cannot understand. By right, we should plug the samples of the marginal posterior (i.e., $\mu | y $) of $\mu$ into the summation $\hat{\phi} = \dfrac{1}{N} \sum_{i=1}^{N} \dfrac{2 \mu_i + y}{y + 1}$, right? How come we can directly use the Gibbs samples of $\mu$ (which is $\mu | \phi, y$)? | |
| Jul 19, 2016 at 4:02 | history | edited | Greenparker | CC BY-SA 3.0 |
added 2 characters in body
|
| Jul 18, 2016 at 22:26 | vote | accept | mscnvrsy | ||
| Jul 18, 2016 at 22:23 | comment | added | mscnvrsy | Okay, I will go through the paper. Thanks for taking the time to help me! | |
| Jul 18, 2016 at 22:21 | comment | added | Greenparker | If you were obtaining independent samples, yes it would minimize the variance of estimators, however, since you are dealing with Markov chains, its generally known that RB does not necessarily reduce variance, and there are some instances where the variance even increases. This paper by Charlie Geyer gave some examples to this point. | |
| Jul 18, 2016 at 22:16 | comment | added | mscnvrsy | Wow, thank you very much for clarifying this to me. So assuming that I know the full conditional distributions, I can work with the theoretical means of the conditional distributions and average over these theoretical means (such as E[phi|mu,y]) to obtain the RB-estimator? This would then minimize the variance of my parameter estimates? | |
| Jul 18, 2016 at 22:12 | comment | added | Greenparker | @mscnvrsy I added an example to help | |
| Jul 18, 2016 at 22:12 | history | edited | Greenparker | CC BY-SA 3.0 |
added 1281 characters in body
|
| Jul 18, 2016 at 21:59 | comment | added | Greenparker | That is incorrect. In Gibbs sampling, you don't know the posterior distribution of the parameter, but know the full conditional posterior for each parameter. There is a big difference between the two. Above, the posterior is $f(\mu, \phi \mid data)$ which is unknown, and for the Gibbs sampler to work you need to know both $f(\mu \mid \phi, data)$ and $f(\phi \mid \mu, data)$. And you are also incorrect in your second understanding. You still need to take a sample from the marginal posterior of $\mu$, and then calculate the sample mean of $\phi^*$ using those samples to find the R-B estimator. | |
| Jul 18, 2016 at 21:43 | comment | added | mscnvrsy | So assuming the posterior distribution of the parameter is known (which to the best of my knowledge happens to be true when applying Gibbs sampling), taking the mean of the distribution rather than a random sample would be the Rao-Blackwellized estimator? I hope I understood your answer correctly. Thank you very much already! | |
| Jul 18, 2016 at 19:25 | history | answered | Greenparker | CC BY-SA 3.0 |