What does the most ideal power curve look like?

What does the worst power curve look like?

Does there exist good power curves or bad power curves?

Obviously there exists more powerful and less powerful hypothesis tests. I'm not sure whether looking at their power curves is useful for ranking their power.

Example of power curve

enter image description here


I recognize that image!

High power is better whenever the null is false, so at a fixed significance level, the steeper the curve rises toward 1 (where it would always reject), the better, and the shallower, the worse it is (the more often you fail to reject when you should).

Here's a sequence of power curves for a two tailed symmetric test, like a two-tailed t-test, say.

sequence of increasingly steep power curves

The flatter curves (which are red-orange in the plot, apart from the totally flat one in grey) are relatively poor -- power is comparatively low while the steeper (bluer) ones are relatively good - power jumps up very near 1 as soon as the difference moves a little distance from the null value.

For a one tailed test you want the rejection rate to not exceed the significance level on the part where the alternative doesn't hold (rejection rate $\leq \alpha$) and then on the part where the alternative does hold, for it to rise as rapidly as possible toward 1.

Ideally, in whichever case, you reject when the null is false, so you would like power to be at 1 everywhere the null is false. This is usually not attainable in practice.

For some situations, there may be a uniformly most powerful test; in particular you might like to read about the Karlin-Rubin theorem. In such a case, for a given set of assumptions and sample size there may be a "best" power curve you can attain under those assumptions across a sequence of alternatives (though you can do better by increasing the sample size).

Even when there isn't a single "best" test, you can compare different tests under some particular sequence of alternatives.

For example, there's plots showing some comparisons of different tests in specific situations (both in question and answer) in this question How to graph Wilcoxon test power R

It is sometimes the case that a power curve dips below the significance level when the alternative is true; in particular this is the case with some omnibus-alternative tests (like goodness of fit tests). Such tests are said to be biased.

  • $\begingroup$ Given the image, Would the most ideal power curve be a flat line at power=1? For all deltas, the power is 1? $\endgroup$ – Numbers Jan 22 at 1:50
  • 1
    $\begingroup$ For all values of the parameter except those specified under the null (always reject the null when it's false is ideal, naturally). However power curves are normally continuous so you can't get there. Hang on and I'll draw some pictures. $\endgroup$ – Glen_b -Reinstate Monica Jan 22 at 1:52
  • 1
    $\begingroup$ Nonparametric tests are NOT less powerful in general. Some nonparametric tests are less powerful than the most powerful parametric test in exactly the situation where the parametric test is most powerful. But even then there's often a nonparametric test that - at the attainable significance levels of the parametric test - may be equally powerful, or very nearly so. As an example, take the t-test; if all its assumptions hold, it's hard to beat (but you can do as well, with a judicious choice of test), and if you modify the situation just a little, you can beat it with a nonparametric test. $\endgroup$ – Glen_b -Reinstate Monica Jan 22 at 2:35
  • 1
    $\begingroup$ It's an image I made in R. For example, I used it here $\endgroup$ – Glen_b -Reinstate Monica Feb 27 at 21:49
  • 1
    $\begingroup$ Also here. OP presumably grabbed it from one of my posts (which is fine), but didn't credit me as the source (which contravenes both the creative commons license terms and the stackexchange rules; unfortunately my comment wasn't enough of a prompt). $\endgroup$ – Glen_b -Reinstate Monica Feb 27 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.