7
$\begingroup$

While I have a vague intuition that this makes sense, I am interested in the formal demonstration that the prior predictive distribution in Bayesian inference is equal to the integral over $\theta$ of the product of the prior distribution $p(\theta)$ and the sampling distribution $p(y|\theta)$, such that:

$$p(y) = \int_{\theta} p(\theta) p(y|\theta)\text{d}\theta.$$

Could one say that the integral makes the distribution unconditional (i.e. it removes the conditionality) by integrating over all possible parameters?

If so, is there a more formal explanation?

$\endgroup$
  • 1
    $\begingroup$ This is not a proof but a definition. This is what we call the prior predictive (or the marginal), as opposed to the posterior predictive$$p(y|y_0)=\int p(\theta|y_0)p(y|\theta)\text{d}\theta,$$where $y_0$ is the observation. $\endgroup$ – Xi'an Dec 31 '14 at 15:28
  • $\begingroup$ Another way of explaining why "predictive": if you want to predict the behaviour of anything related with the random variable associated with $p(y|\theta)$, you need to generate first a value of $\theta$ from the prior $p(\theta)$ before you can generate a realisation $y$ from $p(y|\theta)$. $\endgroup$ – Xi'an Dec 31 '14 at 15:43
6
$\begingroup$

The equation follows from the definition of marginal distribution:

$$ p(y) = \int_\theta{p(y, \theta)} $$

And, from factoring the joint probability of data and parameters into conditional probabilities, like so: $$p(y, \theta) = p(y|\theta)p(\theta)$$

(If this is confusing, divide both sides by $p(\theta)$ to get the familiar definition of conditional probability.)

More plainly, and as referenced in comments, the prior predictive distribution is the Bayesian term defined as the marginal distribution of the data over the prior: It denotes an interpretation of a particular marginal distribution.

$\endgroup$
1
$\begingroup$

Yes it removes the conditionality. Prior Predictive is for predictive a new value BEFORE the sample has been gathered. The only information we have at this stage is our belief about the Prior, $p(\theta$) and sampling distribution i.e. $p(y_{new}|\theta)$.

After the sample has been gathered, we have new information i.e. the likelihood. Hence, now we can predict based on Posterior Predictive Distribution $p(y_{new}|y_{data})$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.