# Proof of integral to give the probability density of new data

Due to the nature of the subject I've found this integral impossible to Google and a quick flick through what books I have has provided nothing. I suspect that the proof is fairly trivial and that my failure to discover it is only due to a lack in both my statistical and searching skills, so I'll just skip the context and get right to asking about. Where $\theta$ are your parameters, $y$ is you data, and $\overset\sim y$ is your new data, how can we prove that $p(\overset\sim y|y) = \int p(\overset\sim y|\theta) p(\theta| y) \ \text{d}\theta$?

• What exactly would you like to prove? If you'd like to prove that it applies to new data, then this is something that you assume rather then prove.
– Tim
Oct 21, 2017 at 18:16
• Plain and simply, that what's on the right hand size of the equation is equal to what's on the left. Oct 21, 2017 at 18:23

This is called posterior predictive distribution in Bayesian statistics.

The assumption for this to hold is that $y$ and $\bar{y}$ are independent conditioned on $\theta$ i.e.

$$y \perp \bar{y} | \theta$$

Thus $p(\bar{y}|\theta, y) = p(\bar{y}|\theta)$. Then we can write

$$p(\bar{y} | y) = \int p(\bar{y}|\theta, y)p(\theta| y)d\theta = \int p(\bar{y}|\theta)p(\theta|y)d\theta$$

We simply marginalize out $\theta$ from joint distribution conditioned on $y$: $p(\bar{y}, \theta | y)$

• I think I must be missing something, so I'm a little afraid to ask, but: why is the conditional independence assumption needed? This equality seems only to be a theorem about conditional probabilities that holds generally and more or less expresses the Axiom of Total Probability.
– whuber
Oct 21, 2017 at 19:13
• Mr whuber, since I have still much to learn your doubts are my doubts :-), but the conditional independence is needed for the last equality to hold (just getting rid on $y$ in $p(\bar{y}|\theta)$ Oct 21, 2017 at 19:26
• Thank you for responding! I figured out my misunderstanding by drawing the DAG; that is, by contemplating the difference between $y\to\theta\to\tilde y$ (conditional independence) and $y\to\tilde y; \theta\to\tilde y$ (no conditional independence). That makes it easy to produce counterexamples when conditional independence does not hold. (+1)
– whuber
Oct 22, 2017 at 15:12