The distribution of a posterior predictive p-value under certain assumptions

Question

I am wondering if anyone can check my understanding of the following passage concerning posterior predictive p-values in the textbook "Bayesian Data Analysis 3rd Edition" on page 151:

In the special case that the parameters $\theta$ are known (or estimated to a very high precision) or in which the test statistic $T(y)$ is ancillary (that is, if it depends only on observed data and if its distribution is independent of the parameters of the model) with a continuous distribution, the posterior predictive p-value $P(T(y^{\text{rep}}) > T(y) \mid y)$ has a distribution that is uniform if the model is true.

I suspect this is something akin to what is called the "CDF transformation" where a random variable $X$ is transformed by plugging it into its own CDF. In other words, if $X$ has a cdf $F_X( \cdot)$, then the random variable $F_X(X)$ is uniformly distributed. And correct me if this is wrong, but another way to write this is $P(X \le x \mid \tilde{X} = x)$ where $\tilde{X}$ is just another iid copy of $X$.

So I guess if we can show that

• $T(y^{\text{rep}}) \mid y$ and
• $T(y) \mid \theta$

are distributed in the same way, that this implies

$$ P(T(y^{\text{rep}}) > T(y) \mid y) = 1 - F(T(y)) $$

where $F$ is the CDF of both of those random variables. Is that right?

Answer 1

Thanks to @Xi'an for the confirmation. The authors are saying that when $$ P(T(y^{\text{rep}}) > T(y) \mid y) = P(T(y^{\text{rep}}) > T(y) \mid \theta, y) $$ then the reasoning of the CDF transformation mentioned above implies that these are uniformly distributed on the unit interval.

I was planning on asking about the next paragraph as well, but I think I understand it now.

More generally, when posterior uncertainty in $\theta$ propagates to the distribution of $T(y \mid \theta)$, the distribution of the p-value is more likely to be near $.5$ than near $0$ or $1$. (To be more precise, the sampling distribution of the p-value has been shown to be 'stochastically less variable' than uniform.)

Perhaps by "stochastically less variable" they mean convex-ordered. If this is the case, it's straightforward to show, using Jensen's inequality, that conditional probabilities are convex-ordered when one conditions on less than the other.