# In a Bayesian MCMC model, if we plug the average of posterior draws back into the Likelihood, would it be estimating the Posterior Predictive?

Suppose we have a Bayesian model with data $$y$$ and a parameter to be estimated, $$\theta$$. Then the posterior is written as:

$$p(\theta | y) \propto p(y|\theta)p(\theta)$$

Suppose that we used an MCMC to obtain draws of the posterior, $$p(\theta | y)$$. Suppose we then take the average of these draws and then plug the average back into the likelihood, $$p(y|\theta)$$, and plot it across the support on $$y$$. Would this distribution be approximating the posterior predictive? Or is it approximating something completely different?

## 2 Answers

The posterior predictive is approximated by the average of the sampling densities over the MCMC sample $$p(y|y^\text{obs})\approx\frac{1}{T}\sum_{t=1}^T p(y|\theta^{(t)})$$ To simulate from this distribution, one need pick one of the $$\theta^{(t)}$$ at random and simulate $$Y$$ from the corresponding $$p(y|\theta^{(t)})$$ [and not for all the $$\theta^{(t)}]$$.

• If I wanted to build the histogram of the simulated distribution, it seems there are two approaches. The first (yours), is randomly picking $\theta^{(t)}$ at random, simulate $Y$, iterate this many times, then plot the $Y$'s I get. The second, seems to be picking $\theta^{(1)}$, simulating $Y^{(1)}$, picking $\theta^{(2)}$, simulating $Y^{(2)}$, etc, until I get $T$ such $Y$'s and getting their histogram. This second one doesn't pick the $\theta^{(t)}$'s randomly, is it the same? What is the purpose of choosing $\theta^{(t)}$ randomly (which I also assume is by replacement)? – user321627 Feb 20 '20 at 2:14
• If you want to draw/plot the approximate distribution, you simply need averaging the $p(y|\theta^{(t)})$'s, there is no need to draw/simulate $Y_i$'s. If you want to exactly draw/simulate from the approximate predictive [rhs], sampling with replacement is needed. – Xi'an Feb 20 '20 at 9:44
• @Xi’an I’m a bit puzzled by “not for all”. Why not do a prediction as the final step of each iteration? I thought this was the standard way. – hejseb Feb 21 '20 at 5:26

No. You'd have* to simulate from the likelihood for each (or at least a representative subset of) MCMC samples to obtain samples from the posterior predictive distribution. Otherwise your uncertainty would not matter - plus many models are highly non-linear.

* There are of course alternative approaches.

• Thanks, would the above method of averaged parameters serve as an approximation? – user321627 Feb 27 '19 at 20:37
• No (unless there is essentially no uncertainty around the parameter). Perhaps sampling from a (possibility multivariate) normal distribution around the mean might work for some suitable transformation (e.g. often for the logit of a single probability). – Björn Feb 27 '19 at 20:41