I'm trying to prove Theorem 10.12 in All of Statistics, which states the following:

Suppose that the size $ \alpha $ test is of the form $$ \text{reject } H_0 \text{ if and only if } T(X^n) \geq c_{\alpha}. $$ Then, $$ \text{p-value} = \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} (T(X^n) \geq T(x^n)) $$ where $ x^n $ is the observed value of $ X^n $. If $ \Theta_0 = \{\theta_0\} $, then $$ \text{p-value} = \mathbb{P}_{\theta_0} (T(X^n) \geq T(x^n)). $$

Wasserman (the author) defines the p-value as $$ \inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \in R_{\alpha} \right\} $$ where $ R_\alpha $ is the rejection region for the hypothesis test. (Note that technically I think this should be $ X \in R_{\alpha} $ given that he defines the rejection region as a subset of the domain of the samples not the range of the test statistic.) He defines a size $ \alpha $ test as a test in which $ \sup_{\theta \in \Theta_0} \beta(\theta) = \alpha $ for power function $ \beta(\alpha) = \mathbb{P}_{\alpha}(X \in R) $.

Thus, to try and prove Theorem 10.12, I started by expanding the definition of p-value in terms of the theorem's assumptions, $$ \text{p-value} = \inf_{\alpha \in (0, 1)} \left\{\sup_{\theta \in \Theta_0} \mathbb{P}_{\theta} \left(T(X^n) \geq c_\alpha\right): T(x^n) \geq c_\alpha \right\}. $$

This is where I hit a dead end though. I tried finding a formula for $ c_\alpha $ that doesn't include $ \alpha $ but got stuck having to invert $ \sup $.

It seems like an alternative way to look at this is to view it as saying that the value of $ c_\alpha $ that minimizes $ \sup_{\theta \in \Theta_0} \mathbb{P}_{\theta}(T(X^n) \geq c_\alpha) $ is the observed value of $ T(X^n) $, $ T(x^n) $. But I'm really uncertain how to prove this.

  • $\begingroup$ Since this is the definition of a p-value, it seems a bit misguided to "prove" it. Instead, just start from where a distribution (like the T-distribution) used to compute p-values comes from. $\endgroup$ Apr 6 '20 at 17:29
  • $\begingroup$ I know this is the definition of a p-value in other texts but here the definition of a p-value is actually the smallest size test that rejects the null and my goal is to prove that that is equivalent to the above definition. $\endgroup$
    – an1lam
    Apr 6 '20 at 17:31
  • $\begingroup$ Rejecting the null is an arbitrary decision. It's made by convention based on a particular p-value. $\endgroup$ Apr 6 '20 at 17:34
  • $\begingroup$ Did you look at what I said above about how Wasserman defines the p-value? He first defines hypothesis tests in terms of hypotheses, power, and size and then defines the p-value as the minimum size test for which we can reject the null hypothesis given some results. $\endgroup$
    – an1lam
    Apr 6 '20 at 17:53
  • 1
    $\begingroup$ I think this can be reworded the following way: prove that a $(1-p)\times 100\%$ confidence interval has the hypothesized value as an endpoint. $\endgroup$
    – Dave
    Apr 6 '20 at 18:57

My solution: we're looking for the smallest $ \alpha $ for which we reject the null hypothesis given our assumptions about the form of the test.

By assumption, for a level $ \alpha $ test, we'll reject the null if and only if $ T(x^n) \geq c_{\alpha} $. Now, note that $ \alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq c_\alpha) $ is a decreasing function of $ c $. Thus, we can recast our goal as maximizing $ c_{\alpha} $ while still rejecting the null.

If we observe $ T(x^n) $, then clearly the maximum value of $ c_{\alpha} $ for which we'll reject the null hypothesis is $ c_{\alpha} = T(x^n) $. A test for which we reject the null if $ T(X^n) \geq T(x^n) $ has size $$ \alpha = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)), $$ so $$ \text{p-value} = \inf_{\alpha \in (0, 1)} \left\{ \alpha: T(X^n) \geq c_\alpha\right\} = \sup_{\theta \in \Theta_0} \mathbb{P}(T(X^n) \geq T(x^n)). $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.