2
$\begingroup$

I'm running a series of Wilcoxon-Mann-Whitney tests using wilcox.test from the stats package in R. I keep getting results where P = 1 for a two-sided test. I have pulled out the simplest case and converted the raw numbers to integer ranks (which gives the same result because this is a ranks-based test):

# unique values from 1 to 14, arranged in 2 sets
x <- c(5, 10, 8)
y <- c(9, 7, 6, 13, 1, 2, 12, 14, 11, 4, 3)
wilcox.test(x, y, alternative = "two.sided")
# for comparison 
wilcox.test(x, y, alternative = "less")
wilcox.test(x, y, alternative = "greater")

I have two questions: (academic) what is happening here? (practical) how can I get a more reasonable P value? The context here is that I'm doing multiple tests and combining P values using Stouffer's method, which doesn't work when P = 1 for one of the results. So, this behavior is ruining my strategy.

I'm pretty sure that what is happening is that I'm repeatedly stumbling on cases where the chance of x being ranked lower than a randomly chosen y is exactly 0.5. If this is true, then simplest case must be

x <- 2
y <- c(1, 3)

and this also gives P = 1. It doesn't matter whether I'm using wilcox.test from the stats package or wilcox_test from the coin package.

$\endgroup$

migrated from stackoverflow.com Nov 16 '15 at 21:05

This question came from our site for professional and enthusiast programmers.

0
$\begingroup$

You are right. For the examples you give, the probability of an observation from x being greater than an observation from y is very close to 0.5.

One statistic to evaluate this is Vargha and Delaney's A, which is the probability of an observation from the first group being greater than an observation from the second group. Cliff's delta is linearly related to this.

if(!require(effsize)){install.packages("effsize")}

x <- c(5, 10, 8)
y <- c(9, 7, 6, 13, 1, 2, 12, 14, 11, 4, 3)

library(effsize)

VD.A(x,y)

   ### Vargha and Delaney A
   ### 
   ### A estimate: 0.5151515 (negligible)

This is so close to 0.50 that you would have to increase your sample size over 500 times to get a significant difference.

X = rep(x, 589)
Y = rep(y, 589)

wilcox.test(X, Y)

   ### Wilcoxon rank sum test with continuity correction
   ### 
   ### data:  X and Y
   ### W = 5897700, p-value = 0.04993
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy