1
$\begingroup$

According to wikipedia, the posterior predicative distribution is

$p(\tilde x | X) = \int p(x | \theta) p(\theta | X) d\theta$

How does one reach at the given equality?

I have

$p(\tilde x | X) = \int p(\tilde x,\theta |X) d\theta = \int p(\tilde x | \theta, X) p(\theta| X) d\theta$ where I use that $p(\tilde x, \theta, X) = p(X) p(\theta|X) p(\tilde x|\theta, X)$ but this is not consistent with the given formula.

Sorry I realized that I made a terrible mistake in the original description after looking at the comment and the answer, and now I have corrected.

$\endgroup$
2
  • $\begingroup$ As α is fixed on both sides, for operational simplicity, ignore it, So p(x) must equal the ∫ p(x|θ)p(θ)dθ but p(x|θ)p(θ) = p(x,θ). So now, we are looking at ∫ p(x,θ))dθ, that is, we are integrating out the θ, which leaves just p(x) (or, more precisely, p(x|α)) as was to be proved. $\endgroup$ – AJKOER Jul 16 '20 at 23:59
  • $\begingroup$ are you saying that $\alpha$ is not a random variable? $\endgroup$ – koch Jul 17 '20 at 0:01
1
$\begingroup$

This is because the variable $\alpha$ only appears in the prior, not the likelihood. So $p(x|\theta,\alpha)=p(x|\theta)$. The term hyper-parameter is used to describe this.

update due to changed question

This property is normally due to conditional independence, where the new data only depends on old data via the common parameters. A classic example is $(y_i|\theta)\sim Bernoulli(\theta)$. If you knew $\theta$ then any other data $y_{i+1},y_{i+2},...$ are irrelevant for inference about $y_i$. So you have say $Pr(y_i=1|\theta)=\theta$. But when you don't know $\theta$ this is when the other data are relevant and useful, because the give information about $\theta$.

$\endgroup$
2
  • $\begingroup$ Thank you for answering. but I do not quite get that. In the case that I encountered, $\alpha$ is some data that can be observed, I cannot see why $p(x|\theta, \alpha) = p(x, \theta)$ from a purely mathematical point of view. Are you saying that $x$ and $\alpha$ are independent? $\endgroup$ – koch Jul 16 '20 at 23:44
  • $\begingroup$ Now that I think about it, they are indeed independent because the data is assumed to be drawn independently, and $x$ is the current observation while $\alpha$ is the previous observation. $\endgroup$ – koch Jul 16 '20 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.