Proving the Formula of the Posterior Predictive Check

I am trying to understand where the formula used in Posterior Predictive Check comes from:

$$P(y^* | y) = \int p(y^*|\theta) p(\theta|y) d\theta$$

Where:

• $$y^*$$ is the new data
• $$y$$ is the original data
• $$\theta$$ are the model parameters
• $$p(\theta |y)$$ is the posterior probability distribution
• $$p(y^* |y)$$ is the probability of observing new data based on the existing data
• $$p(y^* |\theta)$$ is the probability of observing new data based on your model (i.e. the likelihood)

In the above equation, we first generate a random realization of $$\theta_i$$ from the posterior distribution, and then generate a random value of $$y_i$$ from the likelihood's distribution centered around $$\theta_i$$. If the model fits the data well, then in general :

$$P(y^* \mid y) \approx P(y)$$

Here is my attempt to derive the formula used in Posterior Predictive Check:

Step 1: For argument sake, let's write (using the Law of Conditional Probability):

$$P(y^*, \theta \mid y) = \frac{P(y^*, \theta, y)}{P(y)}$$

Step 2: Now, let's focus just on $${P(y^*, \theta, y)}$$.

If we say that $$A =y^*$$ and $$B=\theta, y$$, then:

$$P(A \cap B) = P(A \mid B)P(B)$$ $$P(y^*, \theta , y) = {P(y^* | \theta, y)} \cdot P(\theta, y)$$

Step 3: Going back to Step 1, we can take the result from Step 2 and write:

$$P(y^*, \theta \mid y) = \frac{P(y^*, \theta, y)}{P(y)} = \frac{{P(y^* | \theta, y)} \cdot P(\theta, y)}{P(y)}$$

We can also see that:

$$P(\theta \mid y) = \frac{P(\theta, y)}{P(y)}$$

Thus, we can re-write it as:

$$P(y^*, \theta \mid y) = \frac{P(y^*, \theta, y)}{P(y)} = {P(y^* | \theta, y)} \cdot P(\theta | y)$$

Step 4: Now, we integrate both sides with respect to $$\theta$$:

$$\int P(y^*, \theta \mid y) \, d\theta = \int {P(y^* | \theta, y)} \cdot P(\theta | y) \, d\theta$$

From here, the Left Hand Side loses the integral - thus we can write:

$$P(y^* \mid y) = \int {P(y^* | \theta, y)} \cdot P(\theta | y) \, d\theta$$

Here is where I am stuck.

I don't know how I can further simplify :

$$P(y^* \mid y) = \int {P(y^* | \theta, y)} \cdot P(\theta | y) \, d\theta$$

$$P(y^* \mid y) = \int {P(y^* | \theta)} \cdot P(\theta | y) \, d\theta$$

I read this has to do with a conditional independence assumption, i.e. $$P(y^* | \theta, y) = P(y^* | \theta)$$ ... but I am not quite sure why this assumption is justified in this case.

If you assume the model $$\theta\to y$$ and $$\theta\to y^*$$ then conditional on $$\theta$$, $$y$$ and $$y^*$$ are independent. Therefore $$P(y^*|y,\theta)=P(y^*|\theta)$$
Proof: $$P(y,y^*,\theta)=P(y|\theta)P(y^*|\theta)P(\theta)$$ Conditioning on $$\theta:$$ $$P(y,y^*|\theta)=\frac{P(y|\theta)P(y^*|\theta)P(\theta)}{P(\theta)}=P(y|\theta)P(y^*|\theta)$$