I've been trying to calculate and graph the correspondent power for the wilcoxon signed test and haven't had any luck.

I tried simulating two normal distributed samples, apply the wilcoxon test and finally calculating the p-value, but i'm not sure on how to obtain the graph for it, what I'm looking for is something like this posted by the user Glen_b in thw following question: In what situation would Wilcoxon's Signed-Rank Test be preferable to either t-Test or Sign Test?

enter image description here

Thank you in advance for your help.


In cases where you don't know a way to compute power algebraically, you can use simulation. (That's how the power curves were obtained in the plot you show.)

To compute a single point on the power curve you need a completely specified alternative. In the case of a Wilcoxon-Mann-Whitney test you need to specify the sample sizes, the distribution(s) - you said normal - and then the parameters - the means, and the variances in the case of the normal.

You simulate many samples at a specific alternative and compute the sample rejection rate (in order to approximate the power at that point).

Then you vary one of those specified things to get other points on the curve. In the case in the question (two sample location difference) it's the difference in population means (population locations more generally but I'll keep saying means) we want to look at power against.

In that two sample case case, one mean is held constant (generally at 0 for normals) while the other is varied.

You'll obtain a series of points each of which produces a (binomial) estimate of a probability -- the rejection rate at that point.

See the purple points in the second plot here; plot of power for the one-sample t and signed rank test at the normal with n=4

Normally you'd want to do a few more points, though. For the Wilcoxon-Mann-Whitney, there's an example of power computed at a larger set of points here:

Plot of power at various shifts for normal samples under shift alternative at n=20 in each group

So to compute one of those "W" points - say the one at $d=\frac{\mu_2-\mu_1}{\sigma}=1$), we could simulate the following: generate two independent samples ($n_1=n_2=20$) from normal distributions with means $0$ and $1$ respectively and standard deviation $1$ and perform the hypothesis test. Repeat this some large number ($n_\text{sim}$) of times and compute the proportion of of those times the Wilcoxon-Mann-Whitney test rejected the null, as an estimate of the population rejection rate at this specific alternative.

You would then repeat a simulation like that at each other alternative of interest (i.e. each of the values of $d$ in the above diagram).

Typically you want to end up with a curve rather than a set of points. We have a curve for the t in that first plot above because this is straightforward to calculate at many points without simulation (indeed R has a built-in function for it which was used to produce that curve). Generally we have to simulate and use fewer points.

If you have done many simulations, the standard error will be so small that you can treat it as known (which is why my simulated points don't tend to show standard error bars; they're usually inside the plotted points) and just join the points with a smooth curve; see the last plot at the above link:

power curves for the Shapiro-Wilk and Lilliefors tests of general normality against increasingly skew gamma alternatives

If time permits I generally try to do enough simulations to get the standard error below about a pixel in my display (or even smaller). In practice something below half a percent of the height the plot in a small image is generally plenty; you can probably get away with less precision than that.

If you have done fewer simulations you can still fit a curve using a binomial model (or even weighted least squares may suffice).

There's some discussion of smoothing power curves here How to draw the estimated power curve of a test?. There's a bit of an art to doing it well; with large samples, I often tend to work with $\Phi^{-1}(p)$ when trying to smooth; precisely what I do depends on the circumstances -- often something simple will be sufficient but sometimes more sophisticated approaches are needed to get a good curve. You should take advantage of known facts in producing a curve. [Also see some of the discussion here]


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.