Sum over Posterior Predictive Distribution

Question

I am confused about the posterior predictive distribution.

This is from Murphy's Machine Learning: A Probabilistic Perspective. According to the example set in Chapter 3, the posterior predictive distribution is: $$p(\tilde{x} \in C \mid D) = \displaystyle\sum_{h} p(y = 1 \mid \tilde{x}, h) p(h \mid D) $$ I believe $y=f(x\in C) = \begin{cases} 1 & \quad \text{if } x \in C\\ 0 & \quad \text{if } x \notin C\\ \end{cases} $ although it's not mentioned in the text. Should the posterior predictive distribution sum to 1? $$ \displaystyle\sum_{\tilde{x}} p(\tilde{x} \in C \mid D) = 1$$ or is the interpretation that $$ p(\tilde{x} \in C \mid D) + p(\tilde{x} \notin C \mid D) = 1$$

Answer 1

The last interpretation should be correct. My understanding of the posterior predictive distribution is, that you are looking for the probability of (yet) unobserved data $\tilde{x}$. What you have here is some model with a parameter $h$ and previously observed data which have been processed into a posterior probability distribution for $h$.

It should hold that:

\begin{align*} p(\tilde{x}|D) = \sum_{h}p(\tilde{x},h|D) = \sum_{h}p(\tilde{x}|h)\,p(h|D) \end{align*}

Here $p(\tilde{x}|D)$ is giving the conditional probability for all possible values of $\tilde{x}$. Probability theory states that:

\begin{align*} p(\tilde{x} \in C|D) + p(\tilde{x}\notin C|D) = 1 \end{align*}