4
$\begingroup$

Suppose we have a Bayesian model with data $y$ and a parameter to be estimated, $\theta$. Then the posterior is written as:

$$ p(\theta | y) \propto p(y|\theta)p(\theta) $$

Suppose that we used an MCMC to obtain draws of the posterior, $p(\theta | y)$. Suppose we then take the average of these draws and then plug the average back into the likelihood, $p(y|\theta)$, and plot it across the support on $y$. Would this distribution be approximating the posterior predictive? Or is it approximating something completely different?

$\endgroup$
4
$\begingroup$

The posterior predictive is approximated by the average of the sampling densities over the MCMC sample $$p(y|y^\text{obs})\approx\frac{1}{T}\sum_{t=1}^T p(y|\theta^{(t)})$$ To simulate from this distribution, one need pick one of the $\theta^{(t)}$ at random and simulate $Y$ from the corresponding $p(y|\theta^{(t)})$ [and not for all the $\theta^{(t)}]$.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ If I wanted to build the histogram of the simulated distribution, it seems there are two approaches. The first (yours), is randomly picking $\theta^{(t)}$ at random, simulate $Y$, iterate this many times, then plot the $Y$'s I get. The second, seems to be picking $\theta^{(1)}$, simulating $Y^{(1)}$, picking $\theta^{(2)}$, simulating $Y^{(2)}$, etc, until I get $T$ such $Y$'s and getting their histogram. This second one doesn't pick the $\theta^{(t)}$'s randomly, is it the same? What is the purpose of choosing $\theta^{(t)}$ randomly (which I also assume is by replacement)? $\endgroup$ – user321627 Feb 20 at 2:14
  • $\begingroup$ If you want to draw/plot the approximate distribution, you simply need averaging the $p(y|\theta^{(t)})$'s, there is no need to draw/simulate $Y_i$'s. If you want to exactly draw/simulate from the approximate predictive [rhs], sampling with replacement is needed. $\endgroup$ – Xi'an Feb 20 at 9:44
  • $\begingroup$ @Xi’an I’m a bit puzzled by “not for all”. Why not do a prediction as the final step of each iteration? I thought this was the standard way. $\endgroup$ – hejseb Feb 21 at 5:26
4
$\begingroup$

No. You'd have* to simulate from the likelihood for each (or at least a representative subset of) MCMC samples to obtain samples from the posterior predictive distribution. Otherwise your uncertainty would not matter - plus many models are highly non-linear.

* There are of course alternative approaches.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, would the above method of averaged parameters serve as an approximation? $\endgroup$ – user321627 Feb 27 '19 at 20:37
  • $\begingroup$ No (unless there is essentially no uncertainty around the parameter). Perhaps sampling from a (possibility multivariate) normal distribution around the mean might work for some suitable transformation (e.g. often for the logit of a single probability). $\endgroup$ – Björn Feb 27 '19 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.