Consider a statistical test $\delta$, whose power function is $\pi(\theta|\delta)=\text{Pr}(H_0 \text{ is rejected}|\theta)$. The size of a statistical test $\delta$ is then given as


I understand $\alpha(\delta)$ as a summarization statistic of $\pi(\theta|\delta)$ over $\Theta_0$. It gives an overview of how likely $\delta$ would make type-I error, given that the null hypothesis $H_0$ is true.

However, I don't understand why supremum is preferred over other statistics, e.g. mean, median, etc. in the formulation of $\alpha(\delta)$. For example, we may take

$$\alpha(\delta) = E_{\theta\in\Theta_0}[\pi(\theta|\delta)]$$

Is there any motivation behind using $\sup_{\theta\in\Theta_0}$ for $\alpha(\delta)$?

  • 1
    $\begingroup$ Since we think of $\theta$ has being the true parameter value. Therefore, we are asking what is the probability of committing Type I error. In what you suggest, by using expectation instead, you are instead viewing $\theta$ as following a probability distribution, and hence you are no longer thinking of it as being some fixed quantity. $\endgroup$ Feb 25, 2023 at 6:41
  • $\begingroup$ Thanks for the answer. I'm still confusing about why treating $\theta$ as the true parameter value results in using supremum in the formulation of $\alpha(\delta)$. Are we assuming that the true value of $\theta$ is the worst value it can be, given that we cannot observe that true value? $\endgroup$ Feb 25, 2023 at 7:01
  • 1
    $\begingroup$ The true value of $\theta$ is some number (it is not a random variable). You have no idea what it is. Therefore, you are asking what is the worst case that can happen? $\endgroup$ Feb 25, 2023 at 7:23

1 Answer 1


In this post, I have provided a general framework of hypothesis testing.

The classical loss function associated with hypothesis testing is

\begin{align}L(\theta, a_0) &= \mathbb I_{\Theta_1}(\theta)=\begin{cases}1, &\theta\in\Theta_1,\\0,&\theta\in\Theta_0,\end{cases}\\ L(\theta, a_1) &= \mathbb I_{\Theta_0}(\theta)\end{align}

and the corresponding risk follows as

$$R(\theta,\delta) = \begin{cases}\alpha_\delta(\theta) := \mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_0\\\beta_\delta(\theta) := 1-\mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_1\end{cases}.\tag{1}$$

What do we seek to do in hypothesis testing? Well precisely, we need to find $\delta$ in such a way as to minimize $R(\theta_1,\delta)=1-\mathbb E_{\theta_1}\varphi(\mathbf X),$ for all $\theta_1\in\Theta_1$ subject to the constraint $R(\theta_0,\delta)=\mathbb E_{\theta_0}\varphi(\mathbf X)\leq\alpha$ for all
$\theta_0\in\Theta_0.$ What does the constraint mean? Again writing it: $$\forall\theta\in\Theta_0,~~\mathbb E_\theta\varphi(\mathbf X)\leq \alpha\implies \sup_{\theta\in\Theta_0} \mathbb E_\theta\varphi(\mathbf X)=\alpha.$$ Why the equality in place of $\leq$? Notice $\alpha$ is an upper bound, so the supremum has to be lesser than or equal to $\alpha.$ But the size is defined to be the equality case (as utobi mentioned in the comment, level is defined with $\leq$ constraint).

As for your argument on reasoning that the expectation of a decision rule is a statistic and all that, I couldn't make any sense. But the motivation for why supremum is introduced is given above.

  • 2
    $\begingroup$ (+1) perhaps it may be worth stressing the difference between the size and the level of a test. $\endgroup$
    – utobi
    Feb 25, 2023 at 11:49
  • $\begingroup$ Thanks utobi. Yes, mentioning level would be a good idea to discern the two concepts. $\endgroup$ Feb 25, 2023 at 12:16

This site is temporarily in read-only mode and not accepting new answers.

Not the answer you're looking for? Browse other questions tagged .