1
$\begingroup$

In Chapter 1 of E. L. Lehmann's book Nonparametrics, he refers to the Wilcoxon Rank Sum test in treatment-control experiments. Using Lehmann's notation, let $N$ be the total number of units, $n$ the number of treated units and $m$ the number of control units. Under the null hypothesis of no effects, ranks are fixed over possible assignments to treatment and control and Lehmann provides an exact analytic expression for the expected value and variance of the rank sum statistic (sum of ranks among treated subjects).

Apparently there is no expression for the variance of the Wilcoxon rank sum statistic under the alternative of a larger effect for each of the $N$ units. However, is it possible to show that the Wilcoxon rank sum is consistent against this alternative (i.e., power tends to $1$ as $N \to \infty$)?

To be more precise, let the effect be positive for all units: $\mathbf{y_T}$ is the $N$ outcomes under treatment condition and $\mathbf{y_C}$ is the $N$ outcomes under control condition. Assume that the null of $\mathbf{y_T} = \mathbf{y_C}$ is false and the alternative of $\mathbf{y_T} \geq \mathbf{y_C}$ with a strict inequality holding for at least one unit. If we embed an experiment of $N$ units in an infinite sequence of experiments with increasing $N$, can we show that the probability of rejecting the null hypothesis under the alternative tends to $1$ as $N \to \infty$ (under suitable regularity conditions)? How could we do this formally?

Thanks so much for any insight anyone has to offer.

Improve this question
$\endgroup$
8
  • $\begingroup$ The test is in general only distribution-free under the null, not general alternatives. So you'd usually need to specify the distributions as well. In particular it's not the case for shift alternatives. $\endgroup$
    – Glen_b
    Commented Jul 30, 2021 at 4:32
  • $\begingroup$ With the precise alternative specified, small sample assessment of power is easy to do via simulation. Large sample power properties are usually dealt with via asymptotic arguments, based on one of the notions of efficiency. $\endgroup$
    – Glen_b
    Commented Jul 30, 2021 at 4:44
  • $\begingroup$ Thank you @Glen_b for your insightful response. Would it be correct, then, to say that there are no general results on whether the power of Wilcoxon rank statistic tends to 1 as $N \to \infty$? I'm thinking specifically about power tending to $1$ vis-a-vis the alternative of a larger effect for each unit (which implies that the distribution of the Wilcoxon rank statistic under the alternative is stochastically larger than the statistic under the null). In simulations, it is straightforward to see that power tends to $1$ as $N \to \infty$, but are there no analytical results showing this? $\endgroup$
    – Student_718
    Commented Aug 2, 2021 at 3:37
  • $\begingroup$ No, the test is indeed consistent against a wide variety of alternatives (specifically, ones that change $P(X<Y)$ away from $\frac12$), under mild conditions. There's a difference between not being able to say the exact rate at which power increases (because it depends on the distribution) and being able to say it goes to 1 eventually. $\endgroup$
    – Glen_b
    Commented Aug 2, 2021 at 13:14
  • $\begingroup$ Actually if you add that to your question I think I could expand on the distinction between the power depending on the distribution and consistency in an answer. $\endgroup$
    – Glen_b
    Commented Aug 2, 2021 at 14:30

1 Answer 1

Reset to default
2
$\begingroup$

Under some alternative, the probabilities of the various ranks coming up in a given sample depend on the specific alternative including the distributions under that alternative.

That is, you could have different situations with the same $P(Y>X)$ and so the same expected value of $W$, but not the same distribution of $W$.

e.g. If you were to compare the uniform and the Cauchy for a shift alternative with the same $P(Y>X)$ (a different shift in each case, naturally) and the same sample sizes, and look at the probability that the largest rank was in the $Y$-sample, it would be further from $\frac12$ for the uniform case than in the Cauchy.

However typically this doesn't seem to have a particularly strong effect on the statistic overall.

This doesn't imply that there's any issue with the consistency of the statistic -- it might have some impact on how fast the power goes to $1$ (moving it up or down a bit) but not on the destination.

Mann & Whitney 1947 [1] give a simple proof of consistency for the $P(Y>X)$ case. They give the first moment and bounds on various cross-moments, which is used to bound the variance under the alternative and they then apply Chebyshev's inequality to get lower bound on the rejection probability, which goes to $1$ in the limit.

Lehmann, 1951 [2] shows that the test is unbiased.

[1]: Mann, H., & Whitney, D. (1947). On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other. The Annals of Mathematical Statistics, 18(1), 50-60.

[2]: E. L. Lehmann. (1951) "Consistency and Unbiasedness of Certain Nonparametric Tests." Ann. Math. Statist. 22 (2), 165 - 179

Improve this answer
$\endgroup$
2
  • $\begingroup$ This is excellent. Thank you! $\endgroup$
    – Student_718
    Commented Aug 5, 2021 at 1:46
  • $\begingroup$ I came up with a fairly simple outline of an argument myself but when I checked Mann and Whitney for the details, their own demonstration was simple and made my own effort to motivate the result beside the point. The arguments were quite similar in structure but their own bound was much better (I relied on a much weaker sort of bound) -- I could see such a bound as they give as perhaps having some value in practical use. $\endgroup$
    – Glen_b
    Commented Aug 5, 2021 at 2:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.