# Proof that p-value equals probability of observing equal or more extreme value of test statistic?

I'm trying to prove Theorem 10.12 in All of Statistics, which states the following:

Suppose that the size $$\alpha$$ test is of the form $$\text{reject } H_0 \text{ if and only if } T(X^n) \geq c_{\alpha}.$$ Then, $$\text{p-value} = \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} (T(X^n) \geq T(x^n))$$ where $$x^n$$ is the observed value of $$X^n$$. If $$\Theta_0 = \{\theta_0\}$$, then $$\text{p-value} = \mathbb{P}_{\theta_0} (T(X^n) \geq T(x^n)).$$

Wasserman (the author) defines the p-value as $$\inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \in R_{\alpha} \right\}$$ where $$R_\alpha$$ is the rejection region for the hypothesis test. (Note that technically I think this should be $$X \in R_{\alpha}$$ given that he defines the rejection region as a subset of the domain of the samples not the range of the test statistic.) He defines a size $$\alpha$$ test as a test in which $$\sup_{\theta \in \Theta_0} \beta(\theta) = \alpha$$ for power function $$\beta(\alpha) = \mathbb{P}_{\alpha}(X \in R)$$.

Thus, to try and prove Theorem 10.12, I started by expanding the definition of p-value in terms of the theorem's assumptions, $$\text{p-value} = \inf_{\alpha \in (0, 1)} \left\{\sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} \left(T(X^n) \geq c_\alpha\right): T(x^n) \geq c_\alpha \right\}.$$

This is where I hit a dead end though. I tried finding a formula for $$c_\alpha$$ that doesn't include $$\alpha$$ but got stuck having to invert $$\sup$$.

It seems like an alternative way to look at this is to view it as saying that the value of $$c_\alpha$$ that minimizes $$\sup_{\theta \in \Theta_0} \mathbb{P}_{\theta}(T(X^n) \geq c_\alpha)$$ is the observed value of $$T(X^n)$$, $$T(x^n)$$. But I'm really uncertain how to prove this.

• Since this is the definition of a p-value, it seems a bit misguided to "prove" it. Instead, just start from where a distribution (like the T-distribution) used to compute p-values comes from. Commented Apr 6, 2020 at 17:29
• I know this is the definition of a p-value in other texts but here the definition of a p-value is actually the smallest size test that rejects the null and my goal is to prove that that is equivalent to the above definition. Commented Apr 6, 2020 at 17:31
• Rejecting the null is an arbitrary decision. It's made by convention based on a particular p-value. Commented Apr 6, 2020 at 17:34
• Did you look at what I said above about how Wasserman defines the p-value? He first defines hypothesis tests in terms of hypotheses, power, and size and then defines the p-value as the minimum size test for which we can reject the null hypothesis given some results. Commented Apr 6, 2020 at 17:53
• I think this can be reworded the following way: prove that a $(1-p)\times 100\%$ confidence interval has the hypothesized value as an endpoint.
– Dave
Commented Apr 6, 2020 at 18:57

My solution: we're looking for the smallest $$\alpha$$ for which we reject the null hypothesis given our assumptions about the form of the test.
By assumption from the theorem statement, for a level $$\alpha$$ test, we'll reject the null if and only if $$T(X^n) \geq c_{\alpha}$$. Now, note that $$\alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq c_\alpha)$$ is a decreasing function of $$c_{\alpha}$$. Thus, we can recast our goal as maximizing $$c_{\alpha}$$ while still rejecting the null.
If we observe $$T(x^n)$$, then clearly the maximum value of $$c_{\alpha}$$ for which we'll reject the null hypothesis is $$c_{\alpha} = T(x^n)$$. A test for which we reject the null if $$T(X^n) \geq T(x^n)$$ has size $$\alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)),$$ so $$\text{p-value} = \inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \geq c_\alpha\right\} = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)).$$