Skip to main content
2 of 4
added 1 character in body
sam.b
  • 71
  • 3

Wilcox-Mann-Whitney test giving surprising results

I'm comparing two datasets and want to test the null hypothesis on them. Both datasets contain a lot of meaningful zeros (around 50% of the values are 0). One of them, "test", has more zeros than the other, but the non-zero values are higher. Here are two simplified datasets to illustrate the data that I'm using.

control <- (0, 0, 15, 0, 25, 0, 17, 0, 0, 12, 0)

test <- (0, 0, 0, 0, 30, 0, 19, 0, 0, 20, 0)

I've had trouble figuring out the most appropriate test to use to compare the two datasets, but I've momentarily settled on Mann-Whitney due to the distribution and the high number of zeros (I found some articles saying this test was appropriate for this kind of experiment).

The strange thing that I'm seeing is that occasionally I'll get a significant result, but it appears to be telling me that the group with the lower mean value (zeroes are included when calculating the mean) is actually the group with the smaller sumrank. I even tried checking the rank score for the two datasets manually and found that the one with the higher average mean was lower, as expected. The numbers roughly represent a kind of "revenue per user" number, so I'm definitely looking as average overall revenue as my outcome of interest.

Can anyone help me make sense of the results?? I've also included the R code below that I've ran to discover this (note that the above dataset won't work with the code...that was just a sample)

# This test gives me a p value of .05 when the avg value of test 
# is higher and sumrank is lower

wilcox.test(test, control, paired = FALSE)

# This test gives me a p value of ~.95 with the same dataset
wilcox.test(test, control, paired = FALSE, alternative = 'greater')

I confirmed that the 'greater' alternative works as I'd expect it by running the following code.

# This test gives me a p value of ~0
wilcox.test(test*100, control, paired=FALSE, alternative = 'greater)

Would love any advice that people are willing to give! Thank you!

sam.b
  • 71
  • 3