After some discussion (below), I now have a clearer picture of a focused question, so here is a revised question, though some of the comments might now seem unconnected with the original question.

It seems that t-tests converge quickly for symmetric distributions, that the signed-rank test assumes symmetry, and that, for a symmetric distribution, there is no difference between means/pseudomedians/medians. If so, under what circumstances would a relatively inexperienced statistician find the signed-rank test useful, when s/he has both the t-test and sign test available? If one of my (e.g. social science) students is trying to test whether one treatment performs better than another (by some relatively easily interpreted measure, e.g. some notion of "average" difference), I am struggling to find a place for the signed-rank test, even though it seems to generally be taught, and the sign-test ignored, at my university.

  • 1
    $\begingroup$ Justme: of course, I didn't think about that. $\endgroup$ – JonB Nov 22 '16 at 17:43
  • 2
    $\begingroup$ It depends on whose conventional wisdom you're looking at; my experience of it is very different from yours. Certainly it's easy to find resources that clearly state that symmetry of difference scores is assumed under the null (and that it matters). But note that this is under the null -- as a result, finding lack of symmetry in difference scores in a sample isn't necessarily relevant -- you're not required to have symmetry under the alternative. If you're highly confident that if the null were true the symmetry would hold -- and in many cases it's a highly plausible assumption -- ... ctd $\endgroup$ – Glen_b Nov 23 '16 at 3:10
  • 1
    $\begingroup$ ctd... then there's no issue. The problem is, if you're not prepared to assume it beforehand you don't know if a rejection was caused by assumption failure; the obvious thing to do then is simply not assume it. $\endgroup$ – Glen_b Nov 23 '16 at 3:10
  • 2
    $\begingroup$ Looking your second comment first: (on top of what you already mentioned), note that 1. normal assumptions don't exhaust parametric tests. 2. The signed rank test isn't actually a test of medians but of one-sample Hodges-Lehmann statistics / pseudomedians (though if you add the assumption of symmetry to the alternative, it will also test for medians, and where means exist, also for means, among many other things). Similarly the rank sum test is not a test of medians but of median pairwise differences. You're right that the level of the signed rank test can be quite sensitive to asymmetry. $\endgroup$ – Glen_b Nov 23 '16 at 11:18
  • 2
    $\begingroup$ On your earlier comment: 1 Symmetry isn't generally seen as part of the null, but as part of the assumptions you need in order that the permutations be exchangeable under the null. 2. as previously mentioned, it's not actually a test of medians, but of pseudomedians, and this holds true even under an asymmetric alternative. It's true that interpretation is sometimes easier if you make some restrictive assumptions, but the restrictions required to make it a reasonable test for medians needn't be as strict as assuming symmetry under the alternative. $\endgroup$ – Glen_b Nov 23 '16 at 11:26

Consider a distribution of pair-differences that is somewhat heavier tailed than normal, but not especially "peaky"; then often the signed rank test will tend to be more powerful than the t-test, but also more powerful than the sign test.

For example, at the logistic distribution, the asymptotic relative efficiency of the signed rank test relative to the t-test is 1.097 so the signed rank test should be more powerful than the t (at least in larger samples), but the asymptotic relative efficiency of the sign test relative to the t-test is 0.822, so the sign test would be less powerful than the t (again, at least in larger samples).

As we move to heavier-tailed distributions (while still avoiding overly-peaky ones), the t will tend to perform relatively worse, while the sign-test should improve somewhat, and both sign and signed-rank will outperform the t in detecting small effects by substantial margins (i.e. will require much smaller sample sizes to detect an effect). There will be a large class of distributions for which the signed-rank test is the best of the three.

Here's one example -- the $t_3$ distribution. Power was simulated at n=100 for the three tests, for a 5% significance level. The power for the $t$ test is marked in black, that for the Wilcoxon signed rank in red and the sign test is marked in green. The sign test's available significance levels didn't include any especially near 5% so in that case a randomized test was used to get close to the right significance level. The x-axis is the $\delta$ parameter which represents the shift from the null case (the tests were all two-sided, so the actual power curve would be symmetric about 0).

plot of the power curves for the t, Wilcoxon signed rank and sign tests with n=100 and 5% significance level for the t3

As we see in the plot, the signed rank test has more power than the sign test, which in turn has more power than the t-test.

  • $\begingroup$ Thanks a lot for this @Glen_b ! I'm still struggling to work out where it fits in our syllabus, when we have students for whom even the concept of power is beyond the scope of their studies, and why we teach Wilcoxon as the main alternative to the paired t. But this does give some useful motivations. Thank you! $\endgroup$ – justme Jun 8 '17 at 11:43
  • $\begingroup$ Incidentally after considering what distributional feature impacts the asymptotic variance of the median (and hence the power of the sign test), an example occurred to me where the relative positions of the t and sign test are reversed; as a result I think there's a good possibility of constructing a case where the signed rank test may do considerably better than either of the two other tests. I'll play with it some more when I can and maybe write something on it. $\endgroup$ – Glen_b Jun 8 '17 at 23:04
  • $\begingroup$ As far as your syllabus goes, it's clear there are definitely cases where the signed rank outperforms both other tests (as I outlined in my answers - distributions that are somewhat heavier tailed than normal, but not especially peaked); the t is better at the normal or lighter, and the sign test is better when the distribution has a strong peak (which often tends to go along with very heavy tails, but doesn't have to). [Beware, however, confusing these ideas with mere changes in spread, which doesn't alter their relative properties.] ... I'm sure you could squeeze a few such sentences in $\endgroup$ – Glen_b Jun 8 '17 at 23:30
  • $\begingroup$ Thanks a lot @Glen_b ! The trouble is I'm not teaching the syllabus, just supporting it! The syllabus in most departments seems to be: (i) use a hypothesis test of normality (kill me now) and based on that (ii) either use Wilcoxon or t-Test. So the finer details of the shoulders of the distribution etc are never even touched, and nor is power, just whether assumptions are met (in a slightly rubbish way). But your thoughts are very helpful for me personally, at least! $\endgroup$ – justme Jun 9 '17 at 11:18
  • $\begingroup$ Great post @Glen_b! So in terms of selecting from the two tests, can I conclude that we should always compute power first? Rather than following the assumption that always uses Sign Test if the difference distribution is not normal? Thanks! $\endgroup$ – Lumos May 29 '18 at 6:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.