Skip to main content

Is a property of a hypothesis testing method: the probability of rejecting the null hypothesis given that it is false, i.e. the probability of not making a type II error. The power of a test depends on sample size, effect size, and the significance ($\alpha$) level of the test.

The power of a statistical test $T_n$ is $\inf_{x\in H_1} P_x(T_n = 1)$ and it gives the probability of not making a type II error (rejecting the null hypothesis given that it is false). For simple hypothesis tests the $\inf$ reduces to a point.
Desirable properties for test statistics in general are low level and high power. Unfortunately in most procedures it is not possible to achieve both properties. For instance, whilst it is possible for the t-test to have low level and high power in a large sample setting, in the small sample case it can be shown that its level is $\alpha$ but that the power is at maximum $\alpha$ as well. This follows from the Karlin-Rubin theorem of uniformly most powerful tests.

For a fixed level and power it is possible to calculate the minimum effect size which is required to possibly identify a significant effect. This is known as the minimum detectable distance and it helps in assessing whether in an experiment or a regression it is at all possible to find a significant effect given a desired power.