How to calculate power of different normality tests such as Shapiro-Wilk, Ryan test etc [closed]

Question

Normality test for small samples with power comparison

Answer 1

Goodness of fit tests generally have very broad alternatives. For example, when testing normality, the alternative is usually "anything but normality". To compute the power you need to specify a particular alternative.

Small sample power against a specific alternative can sometimes be calculated algebraically, but the usual approach - one that is both simple and general - is to simulate to obtain the rejection rate.

So, for example, if we want to compute the power of the Shapiro-Wilk test for normality at the exponential distribution, for samples of size 20, then we would carry out the following steps:

repeat nsim times:
   generate an exponential sample of size 20
   compute the Shapiro-Wilk statistic, W
   if (W <= Wcrit) increment the count of rejections
rejectrate = rejections/nsim

You can easily compute an interval for the power (since the sample rejection rate is scaled binomial).

Here's this approach rendered in R (and slightly generalized):

nsim = 1000; alpha = 0.05; n = 20
power = mean(replicate(nsim,(shapiro.test(rexp(n))$p.value<alpha)))

Running that second line several times, I get estimated power of 0.807, 0.823, 0.843. It's a simple matter to change the distribution in that code (or even to write a function and pass in a function to simulate the distribution).

Normally I'd use a larger simulation size. With 100,000 simulations the power in the above example is $0.83402 \pm 0.00231$ (95% interval for the rejection rate).

If you're considering a family of alternatives indexed by a parameter (e.g. the gamma family, indexed by the shape parameter; the scale is irrelevant because the Shapiro Wilk isn't affected by scale), you can plot a power "curve" against the parameter for a number of values (indeed, you could smooth across the calculated rejection rates with a smoothing technique appropriate to binomial proportions and obtain an estimated power curve); alternatively, you could plot against some meaningful function of the parameter (in the case of the gamma, plotting against the skewness coefficient makes a lot of sense, and conveniently places the null at zero rather than infinity).

Here's such a power curve against the gamma family at n=20, alpha=0.05, with nsim of 50000 at each point, with a fairly smooth interpolation (rather than smoothing, since at that sample size the largest standard error is only about half a pixel).

If you're considering power against a fixed alternative but across sample sizes or significance level, again you can obtain a kind of power curve (but against $n$ or the level in this case).

If you consider power against distributions indexed by two shape-related parameters (the Pearson family, for example), or against one shape-parameter and sample size, you will have a power surface or a family of power curves.

If you're just considering a number of unrelated alternatives, generally speaking a table is the most appropriate way to present the information.

When comparing two tests, you can display both on the same plot or in the same table; standard errors should make it easier to see when the differences are more than can be accounted for by random variation (on the other hand, if the power difference is pretty small, it doesn't much matter if you can't tell which is better, because you'd lose little by using the one with lower power).

The Shapiro-Wilk has substantially better power in this case.

(In this case, marking standard error bars is useless, because they would lie inside the spots marking the points and wouldn't be seen.)