Consider a posterior density that involves two parameters: $\beta_1$ and $\beta_2$ given by $f(\beta)$ where $\beta = [\beta_1, \beta_2]^T$. We run a MCMC sampler to sample from the posterior and after the sampler has converged we have samples from the posterior density.

I am interested in generating a random draw $\beta$ from the above posterior using the MCMC samples I have. How can I perform this draw?

  • 1
    $\begingroup$ How did you "account for potential dependence" between parameters? By reparametrizing? Is it not possible to just perform another iteration of Gibb's Sampling? Or if the parameters are independent, can you just sample from their marginal distributions using any of the known ways of sampling from a density? $\endgroup$ – AdamO Jun 12 '18 at 20:24
  • $\begingroup$ Indeed, if the MCMC sampler generates from the correct posterior, one generation will produce the adequate correlation between $\beta_1$ and $\beta_2$. $\endgroup$ – Xi'an Jun 12 '18 at 20:40
  • $\begingroup$ Ignore the dependence issue as that is not pertinent. Suppose that I do a gibbs sampler where I draw from the full conditionals. Simply taking two consecutive draws from the gibbs sampler after convergence seems wrong to me. Is it? $\endgroup$ – user211420 Jun 12 '18 at 21:05
  • 1
    $\begingroup$ @user211420 Are you concerned that taking two consecutive samples is "wrong" because they are correlated samples, or are you specifically interested in obtaining random samples? $\endgroup$ – Greenparker Jun 15 '18 at 8:08
  • $\begingroup$ @Greenparker I know that draws can be correlated and that it is not an issue. I simply want to generate random samples from the posterior. $\endgroup$ – user211420 Jun 15 '18 at 15:36

Assuming $\beta^{(t)}$, the value of the Markov chain after $t$ iterations, is demonstrably generated from the stationary distribution with density $f$, generating$$\beta_1^{(t+1)}|\beta_2^{(t)}\sim f_{1|2}(\beta_1| \beta_2^{(t)})$$produces one random generation $(\beta_1^{(t+1)},\beta_2^{(t)})$ with distribution $f$, since the conditional generation preserves the stationary distribution. So, rigorously,$$(\beta_1^{(t+1)},\beta_2^{(t)}) \sim f$$ For the same reason, generating further$$\beta_2^{(t+1)}|\beta_1^{(t+1)}\sim f_{2|1}(\beta_2| \beta_1^{(t+1)})$$produces one random generation $(\beta_1^{(t+1)},\beta_2^{(t+1)})$ with distribution $f$:$$(\beta_1^{(t+1)},\beta_2^{(t+1)}) \sim f$$with no approximation involved.

What may be the source of your confusion is that, conditional on $(\beta_1^{(t)},\beta_2^{(t)})$, $(\beta_1^{(t+1)},\beta_2^{(t+1)})$ is not distributed from $f$. But marginally $(\beta_1^{(t+1)},\beta_2^{(t+1)})$ is distributed from $f$.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.