I am confused about the posterior predictive distribution.

This is from Murphy's Machine Learning: A Probabilistic Perspective. According to the example set in Chapter 3, the posterior predictive distribution is: $$p(\tilde{x} \in C \mid D) = \displaystyle\sum_{h} p(y = 1 \mid \tilde{x}, h) p(h \mid D) $$ I believe $y=f(x\in C) = \begin{cases} 1 & \quad \text{if } x \in C\\ 0 & \quad \text{if } x \notin C\\ \end{cases} $ although it's not mentioned in the text. Should the posterior predictive distribution sum to 1? $$ \displaystyle\sum_{\tilde{x}} p(\tilde{x} \in C \mid D) = 1$$ or is the interpretation that $$ p(\tilde{x} \in C \mid D) + p(\tilde{x} \notin C \mid D) = 1$$

  • $\begingroup$ While I am not mathematician/statistician enough I believe that using the indicator function might have given nicer notation than using $y=f(x \in C)$? $\endgroup$
    – gwr
    Nov 13, 2015 at 12:41
  • $\begingroup$ I was just reading this text, and the sudden introduction of $y$ also confused me. $\endgroup$ Nov 13, 2015 at 14:33

1 Answer 1


The last interpretation should be correct. My understanding of the posterior predictive distribution is, that you are looking for the probability of (yet) unobserved data $\tilde{x}$. What you have here is some model with a parameter $h$ and previously observed data which have been processed into a posterior probability distribution for $h$.

It should hold that:

\begin{align*} p(\tilde{x}|D) = \sum_{h}p(\tilde{x},h|D) = \sum_{h}p(\tilde{x}|h)\,p(h|D) \end{align*}

Here $p(\tilde{x}|D)$ is giving the conditional probability for all possible values of $\tilde{x}$. Probability theory states that:

\begin{align*} p(\tilde{x} \in C|D) + p(\tilde{x}\notin C|D) = 1 \end{align*}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.