Bayesian statistics: probability of next point

Question

I am reading the Deep Learning book and having some difficulties with the following formula (page 134):

$$ p(X^{m+1} | x^1, \dots, x^m) = \int p(X^{m+1} | \theta) p(\theta | x^1, \dots, x^m) d\theta. $$

Intuitively, it just makes sense: we simply compute the probability of observing the point $m+1$ given the parameters, times the probability of having that parameter given the previous observations. And we integrate over theta because of its uncertainty. Nonetheless, I do not know how to actually derive this formula from other probability rules. Is this just intuition or is there a formal derivation? Thank you.

---

I'm self-studying it, so I'm not asking for help on assignments. Nonetheless, if there is a derivation, I'd like to have just an hint.

Answer 1

Here is some intuition, for any $x,y$ we can write the following: $p(x|y)=\int{p(x,\theta|y)d\theta}$. This is just marginalization. The joint probability inside the integral can also be factored into $p(x|\theta, y)p(\theta|y)$. In your case, $p(x|\theta, y)=p(x|\theta)$ because of the underlying assumption that given the latent parameters your new sample is independent of past data.