# Why is statistical size expressed in terms of supremum?

Consider a statistical test $$\delta$$, whose power function is $$\pi(\theta|\delta)=\text{Pr}(H_0 \text{ is rejected}|\theta)$$. The size of a statistical test $$\delta$$ is then given as

$$\alpha(\delta)=\sup_{\theta\in\Theta_0}\pi(\theta|\delta)$$

I understand $$\alpha(\delta)$$ as a summarization statistic of $$\pi(\theta|\delta)$$ over $$\Theta_0$$. It gives an overview of how likely $$\delta$$ would make type-I error, given that the null hypothesis $$H_0$$ is true.

However, I don't understand why supremum is preferred over other statistics, e.g. mean, median, etc. in the formulation of $$\alpha(\delta)$$. For example, we may take

$$\alpha(\delta) = E_{\theta\in\Theta_0}[\pi(\theta|\delta)]$$

Is there any motivation behind using $$\sup_{\theta\in\Theta_0}$$ for $$\alpha(\delta)$$?

• Since we think of $\theta$ has being the true parameter value. Therefore, we are asking what is the probability of committing Type I error. In what you suggest, by using expectation instead, you are instead viewing $\theta$ as following a probability distribution, and hence you are no longer thinking of it as being some fixed quantity. Commented Feb 25, 2023 at 6:41
• Thanks for the answer. I'm still confusing about why treating $\theta$ as the true parameter value results in using supremum in the formulation of $\alpha(\delta)$. Are we assuming that the true value of $\theta$ is the worst value it can be, given that we cannot observe that true value? Commented Feb 25, 2023 at 7:01
• The true value of $\theta$ is some number (it is not a random variable). You have no idea what it is. Therefore, you are asking what is the worst case that can happen? Commented Feb 25, 2023 at 7:23

In this post, I have provided a general framework of hypothesis testing.

The classical loss function associated with hypothesis testing is

\begin{align}L(\theta, a_0) &= \mathbb I_{\Theta_1}(\theta)=\begin{cases}1, &\theta\in\Theta_1,\\0,&\theta\in\Theta_0,\end{cases}\\ L(\theta, a_1) &= \mathbb I_{\Theta_0}(\theta)\end{align}

and the corresponding risk follows as

$$R(\theta,\delta) = \begin{cases}\alpha_\delta(\theta) := \mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_0\\\beta_\delta(\theta) := 1-\mathbb E_\theta\varphi(\mathbf X), &\theta\in\Theta_1\end{cases}.\tag{1}$$

What do we seek to do in hypothesis testing? Well precisely, we need to find $$\delta$$ in such a way as to minimize $$R(\theta_1,\delta)=1-\mathbb E_{\theta_1}\varphi(\mathbf X),$$ for all $$\theta_1\in\Theta_1$$ subject to the constraint $$R(\theta_0,\delta)=\mathbb E_{\theta_0}\varphi(\mathbf X)\leq\alpha$$ for all
$$\theta_0\in\Theta_0.$$ What does the constraint mean? Again writing it: $$\forall\theta\in\Theta_0,~~\mathbb E_\theta\varphi(\mathbf X)\leq \alpha\implies \sup_{\theta\in\Theta_0} \mathbb E_\theta\varphi(\mathbf X)=\alpha.$$ Why the equality in place of $$\leq$$? Notice $$\alpha$$ is an upper bound, so the supremum has to be lesser than or equal to $$\alpha.$$ But the size is defined to be the equality case (as utobi mentioned in the comment, level is defined with $$\leq$$ constraint).

As for your argument on reasoning that the expectation of a decision rule is a statistic and all that, I couldn't make any sense. But the motivation for why supremum is introduced is given above.

• (+1) perhaps it may be worth stressing the difference between the size and the level of a test. Commented Feb 25, 2023 at 11:49
• Thanks utobi. Yes, mentioning level would be a good idea to discern the two concepts. Commented Feb 25, 2023 at 12:16