# Predicted distribution over the new data sample

I'm reading Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville. In chapter 5 about Bayesian Statistics, to find the distribution over the new data sample $$x^{m+1}$$ after observing $$m$$ samples $$x^{1}, x^{2}, ..., x^{m}$$ we write (equation 5.68 on page 134),

$$p(x^{m+1}|x^{1}, x^{2}, ..., x^{m}) = \int p(x^{m+1}|\theta)p(\theta|x^{1}, x^{2}, ..., x^{m})d\theta$$

How did they derive this ? The only thing I know which connects the data samples ($$x$$'s) and the model parameter ($$\theta$$) is the following

$$p(\theta|x^{1}, x^{2}, ..., x^{m})=\frac{p(x^{1}, x^{2}, ..., x^{m}|\theta)p(\theta)}{p(x^{1}, x^{2}, ..., x^{m})}$$

From the integral in the equation, I think something is being marginalised. But how exactly should this be derived ? I tried applying all the tools that I know (chain rule, Bayes rule), but I could not understand or derive this.

Given this is not the first question I'm posting on math and cross validated SE related to probability (not related to machine learning as such), I do not have a very good understanding of the subject. I do not have the right intuition for such equations involving probabilities and probability distributions. Could you also please explain what is the equation supposed to mean ?

\begin{align*} &p(x^{m+1}|x^{1}, x^{2}, ..., x^{m}) \\ &= \int p(x^{m+1}, \theta|x^{1}, x^{2}, ..., x^{m}) d\theta \tag{defn marginal dstn.}\\ &= \int p(x^{m+1} | \theta, x^{1}, x^{2}, ..., x^{m}) p(\theta|x^{1}, x^{2}, ..., x^{m}) d\theta \tag{defn cndtl prob}\\ &= \int \underbrace{p(x^{m+1}|\theta)}_{(1)}\underbrace{p(\theta|x^{1}, x^{2}, ..., x^{m})}_{(2)}d\theta. \tag{cndtl indep} \end{align*}
• Thanks for taking the time to answer. Is the first step supposed to mean: $\int p((x^{m+1}, \theta)|x^{1}, x^{2}, ..., x^{m})d\theta$ (I'm just adding a bracket for clarity) that is, the joint distribution of $x^{m+1}$ and $\theta$ given $x^{1}, x^{2}, ..., x^{m}$ ? Could you please elaborate the second step ? This is the only definition of conditional probability I know: $P(A|B) = \frac{P(A,B)}{P(B)}$. I don't see: $\frac{p(x^{m+1}, \theta, x^{1}, x^{2}, ..., x^{m})}{x^{1}, x^{2}, ..., x^{m}}$, clearly I misunderstood something. Could you please explain ? I understood the last step! – rranjik Apr 6 at 18:53