Two definitions of p-value: how to prove their equivalence?

Question

I'm reading through Larry Wasserman's book, All of Statistics, and currently about p-values (page 187). Let me first introduce some definitions (I quote):

Definition 1 The power function of a test with rejection region $R$ is defined by $$\beta(\theta)=P_{\theta}(X\in R)$$ The size of a test is defined to be $$\alpha = \sup_{\theta\in\Theta_0}\beta(\theta)$$ A test is said to have level $\alpha$ if its size is less than or equal $\alpha$.

This basically says that $\alpha$, the size is the "biggest" probability of an error of type I. The $p$-value is then defined via (I quote)

Definition 2 Suppose that for every $\alpha\in(0,1)$ we have a size $\alpha$ test with rejection region $R_\alpha$. Then, $$p\text{-value}=\inf\{\alpha:T(X^n)\in R_\alpha\}$$ where $X^n=(X_1,\dots,X_n)$.

For me this means: given a specific $\alpha$ there is a test and rejection region $R_\alpha$ so that $\alpha=\sup_{\theta\in\Theta_{0}(\alpha)}P_\theta(T(X^n)\in R_\alpha)$. For the $p$-value I simply take then the smallest of all these $\alpha$.

Question 1 If this would be the case, then I could clearly choose $\alpha = \epsilon$ for arbitrarily small $\epsilon$. What is my wrong interpretation of definition 2, i.e. what does it exactly mean?

Now Wasserman continuous and states a theorem to have an "equivalent" definition of $p$-value with which I'm familiar (I quote):

Theorem Suppose that the size $\alpha$ test is of the form $$\text{reject } H_0 \iff T(X^n)\ge c_\alpha$$ Then, $$p\text{-value} = \sup_{\theta\in\Theta_0}P_{\theta}(T(X^n)\ge T(x^n))$$ where $x^n$ is the observed value of $X^n$.

So here is my second question:

Question 2 How can I actually prove this theorem? Maybe it's due to my misunderstanding of the definition of the $p$-value, but I can't figure it out.

Answer 1

We have some multivariate data $x$, drawn from a distribution $\mathcal{D}$ with some unknown parameter $\theta$. Note that $x$ are sample outcomes.

We want to test some hypothesis about an unknown parameter $\theta$, the values of $\theta$ under the null hypothesis are in the set $\theta_0$.

In the space of the $X$, we can define a rejection region $R$, and the power of this region $R$ is then defined as $\mathcal{P}_\bar{\theta}^R=P_\bar{\theta}(x \in R)$. So the power is computed for a particular value $\bar{\theta}$ of $\theta$ as the probability that the sample outcome $x$ is in the rejection region $R$ when the value of $\theta$ is $\bar{\theta}$. Obviously the power depends on the region $R$ and on the chosen $\bar{\theta}$.

Definition 1 defines the size of the region $R$ as the supremum of all the values of $\mathcal{P}_\bar{\theta}^R$ for $\bar{\theta}$ in $\theta_0$, so only for values of $\bar{\theta}$ under $H_0$. Obviously this depends on the region, so $\alpha^R=sup_{\bar{\theta} \in \theta_0} \mathcal{P}_\bar{\theta}^R$.

As $\alpha^R$ depends on $R$ we have another value when the region changes, and this is the basis for defining the p-value: change the region, but in such a way that the sample observed value still belongs to the region, for each such region, compute the $\alpha_R$ as defined above and take the infimum: $pv(x)=inf_{R |_{x \in R}} \alpha^R$. So the p-value is the smallest size of all regions that contain $x$.

The theorem is then just a 'translation' of it, namely the case where the regions $R$ are defined using a statistic $T$ and for a value $c$ you define a region $R$ as $R=\{ x | T(x) \ge c \}$. If you use this type of region $R$ in the above reasoning, then the theorem follows.

EDIT because of comments:

@user8: for the theorem; if you define rejection regions as in the theorem, then a rejection region of size $\alpha$ is a set that looks like $R^\alpha= \{X | T(X) \ge c_\alpha \}$ for some $c_\alpha$.

To find the p-value of an observed value $x$, i.e. $pv(x)$ you have to find the smallest region $R$, i.e. the largest value of $c$ such that $\{X | T(X) \ge c \}$ still contains $x$, the latter (the region contains $x$) is equivalent (because of the way the regions are defined) to saying that $c \ge T(x)$, so you have to find the largest $c$ such that $\{X | T(X) \ge c \& c \ge T(x) \}$

Obviously, the largest $c$ such that $ c \ge T(x)$ should be $ c = T(x)$ and then the set supra becomes $\{ X | T(X) \ge c = T(x)\}=\{ X | T(X) \ge T(x)\}$

Answer 2

In Definition 2, the $p$-value of a test statistic is the greatest lower bound of all $\alpha$ such that the hypothesis is rejected for a test of size $\alpha$. Recall that the smaller we make $\alpha$, the less tolerance for Type I error we are allowing, thus the rejection region $R_\alpha$ will also decrease. So (very) informally speaking, the $p$-value is the smallest $\alpha$ we can choose that still lets us reject $H_0$ for the data that we observed. We cannot arbitrarily choose a smaller $\alpha$ because at some point, $R_\alpha$ will be so small that it will exclude (i.e., fail to contain) the event we observed.

Now, in light of the above, I invite you to reconsider the theorem.