5
$\begingroup$

Suppose we have a Bayesian model with data $y$ and a parameter to be estimated, $\theta$. Then the posterior is written as:

$$ p(\theta | y) \propto p(y|\theta)p(\theta) $$

Suppose that we used an MCMC to obtain draws of the posterior, $p(\theta | y)$. Suppose we then take the average of these draws and then plug the average back into the likelihood, $p(y|\theta)$, and plot it across the support on $y$. Would this distribution be approximating the posterior predictive? Or is it approximating something completely different?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
4
$\begingroup$

The posterior predictive is approximated by the average of the sampling densities over the MCMC sample $$p(y|y^\text{obs})\approx\frac{1}{T}\sum_{t=1}^T p(y|\theta^{(t)})$$ To simulate from this distribution, one need pick one of the $\theta^{(t)}$ at random and simulate $Y$ from the corresponding $p(y|\theta^{(t)})$ [and not for all the $\theta^{(t)}]$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ If I wanted to build the histogram of the simulated distribution, it seems there are two approaches. The first (yours), is randomly picking $\theta^{(t)}$ at random, simulate $Y$, iterate this many times, then plot the $Y$'s I get. The second, seems to be picking $\theta^{(1)}$, simulating $Y^{(1)}$, picking $\theta^{(2)}$, simulating $Y^{(2)}$, etc, until I get $T$ such $Y$'s and getting their histogram. This second one doesn't pick the $\theta^{(t)}$'s randomly, is it the same? What is the purpose of choosing $\theta^{(t)}$ randomly (which I also assume is by replacement)? $\endgroup$
    – user321627
    Commented Feb 20, 2020 at 2:14
  • $\begingroup$ If you want to draw/plot the approximate distribution, you simply need averaging the $p(y|\theta^{(t)})$'s, there is no need to draw/simulate $Y_i$'s. If you want to exactly draw/simulate from the approximate predictive [rhs], sampling with replacement is needed. $\endgroup$
    – Xi'an
    Commented Feb 20, 2020 at 9:44
  • $\begingroup$ @Xi’an I’m a bit puzzled by “not for all”. Why not do a prediction as the final step of each iteration? I thought this was the standard way. $\endgroup$
    – hejseb
    Commented Feb 21, 2020 at 5:26
4
$\begingroup$

No. You'd have* to simulate from the likelihood for each (or at least a representative subset of) MCMC samples to obtain samples from the posterior predictive distribution. Otherwise your uncertainty would not matter - plus many models are highly non-linear.

* There are of course alternative approaches.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks, would the above method of averaged parameters serve as an approximation? $\endgroup$
    – user321627
    Commented Feb 27, 2019 at 20:37
  • $\begingroup$ No (unless there is essentially no uncertainty around the parameter). Perhaps sampling from a (possibility multivariate) normal distribution around the mean might work for some suitable transformation (e.g. often for the logit of a single probability). $\endgroup$
    – Björn
    Commented Feb 27, 2019 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.