# Proof that p-value equals probability of observing equal or more extreme value of test statistic?

I'm trying to prove Theorem 10.12 in All of Statistics, which states the following:

Suppose that the size $$\alpha$$ test is of the form $$\text{reject } H_0 \text{ if and only if } T(X^n) \geq c_{\alpha}.$$ Then, $$\text{p-value} = \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} (T(X^n) \geq T(x^n))$$ where $$x^n$$ is the observed value of $$X^n$$. If $$\Theta_0 = \{\theta_0\}$$, then $$\text{p-value} = \mathbb{P}_{\theta_0} (T(X^n) \geq T(x^n)).$$

Wasserman (the author) defines the p-value as $$\inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \in R_{\alpha} \right\}$$ where $$R_\alpha$$ is the rejection region for the hypothesis test. (Note that technically I think this should be $$X \in R_{\alpha}$$ given that he defines the rejection region as a subset of the domain of the samples not the range of the test statistic.) He defines a size $$\alpha$$ test as a test in which $$\sup_{\theta \in \Theta_0} \beta(\theta) = \alpha$$ for power function $$\beta(\alpha) = \mathbb{P}_{\alpha}(X \in R)$$.

Thus, to try and prove Theorem 10.12, I started by expanding the definition of p-value in terms of the theorem's assumptions, $$\text{p-value} = \inf_{\alpha \in (0, 1)} \left\{\sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} \left(T(X^n) \geq c_\alpha\right): T(x^n) \geq c_\alpha \right\}.$$

This is where I hit a dead end though. I tried finding a formula for $$c_\alpha$$ that doesn't include $$\alpha$$ but got stuck having to invert $$\sup$$.

It seems like an alternative way to look at this is to view it as saying that the value of $$c_\alpha$$ that minimizes $$\sup_{\theta \in \Theta_0} \mathbb{P}_{\theta}(T(X^n) \geq c_\alpha)$$ is the observed value of $$T(X^n)$$, $$T(x^n)$$. But I'm really uncertain how to prove this.

• Since this is the definition of a p-value, it seems a bit misguided to "prove" it. Instead, just start from where a distribution (like the T-distribution) used to compute p-values comes from. Apr 6 '20 at 17:29
• I know this is the definition of a p-value in other texts but here the definition of a p-value is actually the smallest size test that rejects the null and my goal is to prove that that is equivalent to the above definition. Apr 6 '20 at 17:31
• Rejecting the null is an arbitrary decision. It's made by convention based on a particular p-value. Apr 6 '20 at 17:34
• Did you look at what I said above about how Wasserman defines the p-value? He first defines hypothesis tests in terms of hypotheses, power, and size and then defines the p-value as the minimum size test for which we can reject the null hypothesis given some results. Apr 6 '20 at 17:53
• I think this can be reworded the following way: prove that a $(1-p)\times 100\%$ confidence interval has the hypothesized value as an endpoint.
– Dave
Apr 6 '20 at 18:57

My solution: we're looking for the smallest $$\alpha$$ for which we reject the null hypothesis given our assumptions about the form of the test.
By assumption, for a level $$\alpha$$ test, we'll reject the null if and only if $$T(x^n) \geq c_{\alpha}$$. Now, note that $$\alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq c_\alpha)$$ is a decreasing function of $$c$$. Thus, we can recast our goal as maximizing $$c_{\alpha}$$ while still rejecting the null.
If we observe $$T(x^n)$$, then clearly the maximum value of $$c_{\alpha}$$ for which we'll reject the null hypothesis is $$c_{\alpha} = T(x^n)$$. A test for which we reject the null if $$T(X^n) \geq T(x^n)$$ has size $$\alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)),$$ so $$\text{p-value} = \inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \geq c_\alpha\right\} = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)).$$