Consider a statistical test $\delta$, whose power function is $\pi(\theta|\delta)=\text{Pr}(H_0 \text{ is rejected}|\theta)$. The size of a statistical test $\delta$ is then given as


I understand $\alpha(\delta)$ as a summarization statistic of $\pi(\theta|\delta)$ over $\Theta_0$. It gives an overview of how likely $\delta$ would make type-I error, given that the null hypothesis $H_0$ is true.

However, I don't understand why supremum is preferred over other statistics, e.g. mean, median, etc. in the formulation of $\alpha(\delta)$. For example, we may take

$$\alpha(\delta) = E_{\theta\in\Theta_0}[\pi(\theta|\delta)]$$

Is there any motivation behind using $\sup_{\theta\in\Theta_0}$ for $\alpha(\delta)$?

  • 1
    $\begingroup$ Since we think of $\theta$ has being the true parameter value. Therefore, we are asking what is the probability of committing Type I error. In what you suggest, by using expectation instead, you are instead viewing $\theta$ as following a probability distribution, and hence you are no longer thinking of it as being some fixed quantity. $\endgroup$ Commented Feb 25, 2023 at 6:41
  • $\begingroup$ Thanks for the answer. I'm still confusing about why treating $\theta$ as the true parameter value results in using supremum in the formulation of $\alpha(\delta)$. Are we assuming that the true value of $\theta$ is the worst value it can be, given that we cannot observe that true value? $\endgroup$ Commented Feb 25, 2023 at 7:01
  • 1
    $\begingroup$ The true value of $\theta$ is some number (it is not a random variable). You have no idea what it is. Therefore, you are asking what is the worst case that can happen? $\endgroup$ Commented Feb 25, 2023 at 7:23

1 Answer 1


In this post, I have provided a general framework of hypothesis testing.

The classical loss function associated with hypothesis testing is

\begin{align}L(\theta, a_0) &= \mathbb I_{\Theta_1}(\theta)=\begin{cases}1, &\theta\in\Theta_1,\\0,&\theta\in\Theta_0,\end{cases}\\ L(\theta, a_1) &= \mathbb I_{\Theta_0}(\theta)\end{align}

and the corresponding risk follows as

$$R(\theta,\delta) = \begin{cases}\alpha_\delta(\theta) := \mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_0\\\beta_\delta(\theta) := 1-\mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_1\end{cases}.\tag{1}$$

What do we seek to do in hypothesis testing? Well precisely, we need to find $\delta$ in such a way as to minimize $R(\theta_1,\delta)=1-\mathbb E_{\theta_1}\varphi(\mathbf X),$ for all $\theta_1\in\Theta_1$ subject to the constraint $R(\theta_0,\delta)=\mathbb E_{\theta_0}\varphi(\mathbf X)\leq\alpha$ for all
$\theta_0\in\Theta_0.$ What does the constraint mean? Again writing it: $$\forall\theta\in\Theta_0,~~\mathbb E_\theta\varphi(\mathbf X)\leq \alpha\implies \sup_{\theta\in\Theta_0} \mathbb E_\theta\varphi(\mathbf X)=\alpha.$$ Why the equality in place of $\leq$? Notice $\alpha$ is an upper bound, so the supremum has to be lesser than or equal to $\alpha.$ But the size is defined to be the equality case (as utobi mentioned in the comment, level is defined with $\leq$ constraint).

As for your argument on reasoning that the expectation of a decision rule is a statistic and all that, I couldn't make any sense. But the motivation for why supremum is introduced is given above.

  • 2
    $\begingroup$ (+1) perhaps it may be worth stressing the difference between the size and the level of a test. $\endgroup$
    – utobi
    Commented Feb 25, 2023 at 11:49
  • $\begingroup$ Thanks utobi. Yes, mentioning level would be a good idea to discern the two concepts. $\endgroup$ Commented Feb 25, 2023 at 12:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.