# proof of the the prior predicative distribution

According to wikipedia, the posterior predicative distribution is

$$p(\tilde x | X) = \int p(x | \theta) p(\theta | X) d\theta$$

How does one reach at the given equality?

I have

$$p(\tilde x | X) = \int p(\tilde x,\theta |X) d\theta = \int p(\tilde x | \theta, X) p(\theta| X) d\theta$$ where I use that $$p(\tilde x, \theta, X) = p(X) p(\theta|X) p(\tilde x|\theta, X)$$ but this is not consistent with the given formula.

Sorry I realized that I made a terrible mistake in the original description after looking at the comment and the answer, and now I have corrected.

• As α is fixed on both sides, for operational simplicity, ignore it, So p(x) must equal the ∫ p(x|θ)p(θ)dθ but p(x|θ)p(θ) = p(x,θ). So now, we are looking at ∫ p(x,θ))dθ, that is, we are integrating out the θ, which leaves just p(x) (or, more precisely, p(x|α)) as was to be proved. – AJKOER Jul 16 '20 at 23:59
• are you saying that $\alpha$ is not a random variable? – koch Jul 17 '20 at 0:01

This is because the variable $$\alpha$$ only appears in the prior, not the likelihood. So $$p(x|\theta,\alpha)=p(x|\theta)$$. The term hyper-parameter is used to describe this.
This property is normally due to conditional independence, where the new data only depends on old data via the common parameters. A classic example is $$(y_i|\theta)\sim Bernoulli(\theta)$$. If you knew $$\theta$$ then any other data $$y_{i+1},y_{i+2},...$$ are irrelevant for inference about $$y_i$$. So you have say $$Pr(y_i=1|\theta)=\theta$$. But when you don't know $$\theta$$ this is when the other data are relevant and useful, because the give information about $$\theta$$.
• Thank you for answering. but I do not quite get that. In the case that I encountered, $\alpha$ is some data that can be observed, I cannot see why $p(x|\theta, \alpha) = p(x, \theta)$ from a purely mathematical point of view. Are you saying that $x$ and $\alpha$ are independent? – koch Jul 16 '20 at 23:44
• Now that I think about it, they are indeed independent because the data is assumed to be drawn independently, and $x$ is the current observation while $\alpha$ is the previous observation. – koch Jul 16 '20 at 23:55