Let $\mathcal{H}_0:\theta\in\Theta_0$ and $\mathcal{H}_1:\theta\in\Theta_1$ be the null and alternative hypotheses. The power of a test $\delta$ is defined as: $$\pi_{\delta}:\Theta\to[0,1] \ s.t.\ \pi_{\delta}(\theta)=\mathbb{P}_{\theta}(\delta(X_1,...,X_n)=1)$$where $\delta(X_1,...,X_n)=1$ if it rejects $\mathcal{H}_0$ and 0 if it doesn't. Also, the size of the test $\delta$ is defined as: $$\alpha_{\delta}^*=\sup_{\theta\in\Theta_0}\pi_{\delta}(\theta)$$ I kind of understand these two concepts mathematically, but they're a bit dry to me yet. Does someone has an intuitive explanation for these concepts, or nice examples, interesting caveats, non-trivial but important remarks, etc. I really would like to have these concepts understood in a deeper way. Thanks in advance.
1 Answer
I think this is a case when the notation makes it much denser than it really is. $\pi_\delta(\theta)$ gives the probability of rejecting $\mathcal H_0$ given that the "truth" is $\theta$, and the size of the test is like the largest probability of rejecting given that $\mathcal H_0$ is true (hence the supremum only being over $\theta\in\Theta_0$). Also I think what you've written is the type I error rate not the power, since the power would be the probability of correctly rejecting when $\theta\in\Theta_1$.
We're only talking about varying $\theta$ here so the event $\{\delta(\vec X) = 1\}$ is fixed, so we could consider $F := \{x \in \mathbb R^n : \delta(x) = 1\}$ (or we could pull all the way back to the original sample space $\Omega$) and then $\pi_\delta$ gives us the size of this event for different $\theta$. And the size of the test then is the biggest measure we give to this event when we only consider $\theta\in\Theta_0$ (so the term "size" makes sense too).
I think this is a bit reminiscent of the difference between loss and risk in that $\pi_\delta(\theta)$ is a useful thing to know for one particular $\theta$, but we need to make a decision at the level of $\Theta_0$ which can contain uncountably many $\theta$, so we want to reduce this to a univariate summary (to take advantage of the total ordering of $\mathbb R$). Often the two candidates for doing this are integrating or optimizing and in this case optimizing turns out to be useful and intuitive (and the collection $\{\pi_\delta(\theta) : \theta\in\Theta_0\}\subset \mathbb R$ so the least upper bound property of $\mathbb R$ means we don't have to worry about the $\sup$ being well-defined).
One consequence of using the $\sup$ is that we only care about the largest $F$ can be. It doesn't matter if $F$ has a small measure for "most" $\theta\in\Theta_0$ (i.e. for most $\theta\in\Theta_0$ we are unlikely to mistakenly reject); if there are any $\theta\in\Theta_0$ that lead to a large probability of rejection then the test has a large size (i.e. type I error rate). This is a worst-case thing.
