Let $\theta$ be the set of parameters for a Bayesian Network model. The model observes some data $D$. I want to then predict the values of some new data $D$. Why is the following true?

$$ p(D'|D) = \int p(D'|\theta)p(\theta|D) d\theta $$

Does it stem from some manipulation of Bayes rule? Or the formula for expected value?


Assuming the dependency of $D'$ on $D$ is only through $\theta$, then $p(D' | \theta) = p(D' | \theta, D)$.

Now you can see all probabilities on the right hand side are simply conditional on $D$: $$ p(D' | D) = \int p(D' | \theta, D) p(\theta | D) d\theta) $$ This behaves exactly as if the condition on $D$ wasn't there, so for educational purposes, let's just forget about it: $$ p(D') = \int p(D' | \theta) p(\theta) d\theta) = \int p(D', \theta) d\theta $$ This last equation just shows the default way of integrating out one variable from a common distribution. As indicated above, this just works the same way if all distributions are conditional.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.