# Two-sample one-sided Kolmogorov-Smirnov test vs one-sided Wilcoxon-Mann-Whitney test

I've read in the library manual for test ks.test that

The possible values "two.sided", "less" and "greater" of alternative specify the null hypothesis that the true distribution function of x is equal to, not less than or not greater than the hypothesized distribution function (one-sample case) or the distribution function of y (two-sample case), respectively. This is a comparison of cumulative distribution functions, and the test statistic is the maximum difference in value, with the statistic in the "greater" alternative being D^+ = max[F_x(u) - F_y(u)]. Thus in the two-sample case alternative = "greater" includes distributions for which x is stochastically smaller than y (the CDF of x lies above and hence to the left of that for y), in contrast to t.test or wilcox.test.

Unfortunately I failed to understand this difference between (I guess one sided) wilcox.test and ks.test. It seems, that both test for displacement of one distribution versus another. Does anyone can shed some light on it, please?

-

Both are testing for displacement of the x variable with respect to the y variable, but the 2 tests have opposite meanings for the term "greater" (and therefor also or "less").

In the ks.test "greater" means that the CDF of 'x' is higher than the CDF of 'y' which means that things like the mean and the median will be smaller values in 'x' than in 'y' if the CDF of 'x' is "greater" than the CDF of 'y'. In 'wicox.test' and 't.test' the mean, median, etc. will be greater in 'x' than in 'y' if you believe that the alternative of "greater" is true.

An example from R:

> x <- rnorm(25)
> y <- rnorm(25, 1)
>
> ks.test(x,y, alt='greater')

Two-sample Kolmogorov-Smirnov test

data:  x and y
D = 0.6, p-value = 0.0001625
alternative hypothesis: two-sided

> wilcox.test( x, y, alt='greater' )

Wilcoxon rank sum test

data:  x and y
W = 127, p-value = 0.9999
alternative hypothesis: true location shift is greater than 0

> wilcox.test( x, y, alt='less' )

Wilcoxon rank sum test

data:  x and y
W = 127, p-value = 0.000101
alternative hypothesis: true location shift is less than 0


Here I generated 2 samples from a normal distribution, both with sample size 25 and standard deviation of 1. The x variable comes from a distribution of mean 0 and the y variable from a distribution of mean 1. You can see the results of ks.test give a very significant result testing in the "greater" direction even though x has the smaller mean, this is because the CDF of x is above that of y. The wilcox.test function shows lack of significance in the "greater" direction, but similar level of significance in the "less" direction.

Both tests are different approaches to testing the same idea, but what "greater" and "less" mean to the 2 tests are different (and conceptually opposite).

-
Thank you! Do you happen to know, which test has better power? I guess it is Wilcoxon-Mann-Whitney, but I don't know how to support it. –  Adam Ryczkowski Oct 15 '12 at 18:24
Which has greater power depends on the shapes of the distributions and how they differ, otherwise why would both still be commonly used? –  Greg Snow Oct 15 '12 at 18:29
That's interesting. Can you please tell me, how does the relative power of both tests depend on both distributions (or at least point me to a literature)? –  Adam Ryczkowski Oct 15 '12 at 18:35
From my experience, the fact that something is commonly used in statistics does not necessary imply it is the correct procedure... –  Adam Ryczkowski Oct 15 '12 at 18:37
Just look at how the test statistic works. The ks test looks at the distance between the ECDF's and the wilcox test looks at the probability of a getting a larger value from y than x when choosing one value at random from each. Find distributions that show a large difference one way, but no the other and you will see large differences in power. If one procedure were clearly dominated by the other, and it was easy to show/explaine that, then I would expect it to fall out of disfavor. This does not say that either is "correct". –  Greg Snow Oct 15 '12 at 19:22