3
$\begingroup$

Due to the nature of the subject I've found this integral impossible to Google and a quick flick through what books I have has provided nothing. I suspect that the proof is fairly trivial and that my failure to discover it is only due to a lack in both my statistical and searching skills, so I'll just skip the context and get right to asking about. Where $\theta$ are your parameters, $y$ is you data, and $\overset\sim y$ is your new data, how can we prove that $p(\overset\sim y|y) = \int p(\overset\sim y|\theta) p(\theta| y) \ \text{d}\theta$?

$\endgroup$
2
  • 1
    $\begingroup$ What exactly would you like to prove? If you'd like to prove that it applies to new data, then this is something that you assume rather then prove. $\endgroup$
    – Tim
    Oct 21, 2017 at 18:16
  • $\begingroup$ Plain and simply, that what's on the right hand size of the equation is equal to what's on the left. $\endgroup$
    – J. Min
    Oct 21, 2017 at 18:23

1 Answer 1

5
$\begingroup$

This is called posterior predictive distribution in Bayesian statistics.

The assumption for this to hold is that $y$ and $\bar{y}$ are independent conditioned on $\theta$ i.e.

$$y \perp \bar{y} | \theta$$

Thus $p(\bar{y}|\theta, y) = p(\bar{y}|\theta)$. Then we can write

$$p(\bar{y} | y) = \int p(\bar{y}|\theta, y)p(\theta| y)d\theta = \int p(\bar{y}|\theta)p(\theta|y)d\theta$$

We simply marginalize out $\theta$ from joint distribution conditioned on $y$: $p(\bar{y}, \theta | y)$

$\endgroup$
3
  • 2
    $\begingroup$ I think I must be missing something, so I'm a little afraid to ask, but: why is the conditional independence assumption needed? This equality seems only to be a theorem about conditional probabilities that holds generally and more or less expresses the Axiom of Total Probability. $\endgroup$
    – whuber
    Oct 21, 2017 at 19:13
  • $\begingroup$ Mr whuber, since I have still much to learn your doubts are my doubts :-), but the conditional independence is needed for the last equality to hold (just getting rid on $y$ in $p(\bar{y}|\theta)$ $\endgroup$ Oct 21, 2017 at 19:26
  • 2
    $\begingroup$ Thank you for responding! I figured out my misunderstanding by drawing the DAG; that is, by contemplating the difference between $y\to\theta\to\tilde y$ (conditional independence) and $y\to\tilde y; \theta\to\tilde y$ (no conditional independence). That makes it easy to produce counterexamples when conditional independence does not hold. (+1) $\endgroup$
    – whuber
    Oct 22, 2017 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.