If we take estimated parameters from an MCMC and plug it back into the likelihood to draw new observations, what does the histogram approximate?

Question

Suppose we have the following set-up and we conduct an MCMC on it.

Likelihood:

$$ X\sim ~ Gamma(\alpha,\beta) $$

Prior:

$$ \alpha \sim Unif(0,10) $$ $$ \beta \sim Gamma(0.5,0.5) $$

Assume the MCMC over $M$ iterations gives us estimates of $\alpha$ and $\beta$ where $\widehat{\alpha} = (\alpha_1, \ldots, \alpha_M)$ and $\widehat{\beta} = (\beta_1, \ldots, \beta_M)$.

Assume we iteratively for $i = \{1, \ldots, M\}$:

• Take $\alpha_i$ and $\beta_i$ and sample a value of $x_i \sim Gamma(\alpha_i,\beta_i)$

If we plot the density of these $(x_1, \ldots, x_M)$, what is it approximating? Is it the posterior predictive?

Answer 1

Yes, this is called the posterior predictive distribution. Mathematically, the histogram is approximating the following distribution

$$ p(\tilde{y} \vert y) = \int p(\tilde{y} \vert \theta) p(\theta \vert y) \, d\theta$$

You'll note the first part of the integrand is the likelihood and the second is the posterior. This integral is integrating over all $\theta$ drawn from the posterior and plugged into the likelihood.