7
$\begingroup$

I am looking at a success of textual requests. We have a dataset of matched pairs, one successful, one not successful, and they are matched based on length of the request in words (since I want to eliminate this effect). I want to determine which features are significant for success.

For some features, e.g. the number of posts of the same user before this request, I get very different significance results for a Mann Whitney U and a Wilcoxon Signed Rank Test (I am using Python/SciPy.stats for this):

Mann Whitney U, 5.91e-6 (one sided)

Wilcoxon signed Rank, 1.4e-2 (two sided)

Why is that? I am not a statistician but I am surprised by this result. "Mann-Whitney U and Wilcoxon Matched pairs are basically the same in that they compare between two medians to suggest whether both samples come from the same population or not." from http://www.le.ac.uk/bl/gat/virtualfc/Stats/nonpcom.html

What assumptions am I missing or what explains this this gap?

Improve this question
$\endgroup$
4
  • $\begingroup$ One is for paired data, the other is not; that's the crucial difference. Neither is actually a comparison of population medians (at least not without additional assumptions) $\endgroup$
    – Glen_b
    Commented Nov 12, 2013 at 21:35
  • 1
    $\begingroup$ Mann-Whitney is for independent groups and uses only ordinal information. Wilcoxon is for matched groups and uses interval information (the pairwise differences). There is no reason to expect the two analyses to give similar results. $\endgroup$
    – Ray Koopman
    Commented Nov 13, 2013 at 0:31
  • $\begingroup$ Also, neither test compares medians per se. Strictly speaking, they are tests of stochastic dominance. With additional assumptions—(1) distributions of identical shape in both groups, and (2) differences in distributions that are only differences in central location—they can be interpreted as tests of median difference. See also my response here stats.stackexchange.com/questions/65735/… $\endgroup$
    – Alexis
    Commented Apr 26, 2014 at 17:54
  • 1
    $\begingroup$ Nobody seems to be mentioning that the OP's confusion probably came from mistaking "Wilcoxon signed rank test" for "Wilcoxon rank sum test". They are different, and only the latter is equivalent to Mann-Whitney U. $\endgroup$
    – jez
    Commented May 8, 2020 at 17:37

1 Answer 1

Reset to default
6
$\begingroup$

In scipy.stats, the Mann-Whitney U test compares two populations:

Computes the Mann-Whitney rank test on samples x and y.

but the Wilcoxon test compares two PAIRED populations:

The Wilcoxon signed-rank test tests the null hypothesis that two related paired samples come from the same distribution. In particular, it tests whether the distribution of the differences x - y is symmetric about zero. It is a non-parametric version of the paired T-test.

EDITED / CORRECTED in response to ttnphns' comments.

Note that the t does not test for whether the distribution of the differences is symmetric about zero, so the Wilcoxon signed rank test is not truly a non-parametric counterpart of the paired t test.

The Mann-Whitney test, on the other hand, assumes that all the observations are independent of each other (no basis for pairing here!). It also assumes that the two distributions are the same, and the alternative is that one is stochastically greater than the other. If we make the additional assumption that the only difference between the two distributions is their location, and the distributions are continuous, then "stochastically greater than" is equivalent to such statements as "the medians are different", so you can, with the extra assumption(s), interpret it that way.

The Mann-Whitney uses a continuity correction by default, but the Wilcoxon doesn't.

The Mann-Whitney handles ties using the midrank, but the Wilcoxon offers three options for handling ties in the paired values (i.e., zero difference between the two elements of the pair.)

It sounds like the Wilcoxon test is the more appropriate for your purposes, since you do have that lack of independence between all observations. However, one might imagine that requests with similar, but not equal, lengths might exhibit similar behavior, whereas the Wilcoxon would assume that if they aren't paired, they are independent. A logistic regression model might serve you better in this case.

Quotes are from the scipy.stats doc pages, which we aren't supposed to link to, apparently.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Thank you very much for the comprehensive answer! I am wondering how one would explain the big discrepancy between these two p values. How can the Wilcoxon that apparently has the additional assumption of pairs be much less sure about the significance? Does it have smaller "statistical power"? So I shouldn't be using Mann Whitney U since my matched pairs are not independent? Within a population they should be. $\endgroup$
    – Tim
    Commented Nov 12, 2013 at 22:08
  • $\begingroup$ It could be the continuity correction, it could be the different way of handling ties, or just the data itself. Without looking at the data, it's really hard to say. How large a sample size do you have? Also remember Gelman's famous dictum: the difference between statistical significance and insignificance is not itself significant. The issue with the matched pairs is that Mann-Whitney assumes everything is independent of everything else, between groups as well as within groups, but w/ matched pairs they aren't independent between groups. $\endgroup$
    – jbowman
    Commented Nov 12, 2013 at 23:12
  • 2
    $\begingroup$ jbowman, I think that the documentation is right and you are wrong. Wilcoxon tests that the differences are symmetric about 0 (= equal prob. for the sum of two randomly chosen differences to be "+" or "-"). It is sign test tests that the distribution of the differences has median 0 (= equal prob. for two randomly chosen differences to be of same or opposite sign). Like any nonparametric, Wilcoxon itself makes no distributional assumptions. However you may add that symmetry-of-differences assumption to narrow it to make inference specifically about the mean (or median) difference. $\endgroup$
    – ttnphns
    Commented Nov 13, 2013 at 9:01
  • 2
    $\begingroup$ P.S. The similar controversy about (actually additional, not intrinsic) assumptions exist over Mann-Whitney or Kruskal-Wallis tests: see e.g. stats.stackexchange.com/q/76059/3277 with all the comments and especially @Glen_b answer. And, btw, Mann-Whitney's H0 is not "distributions are the same" but rather that there's no stochastical dominance (aka the so called "location" is the same). $\endgroup$
    – ttnphns
    Commented Nov 13, 2013 at 10:12
  • 1
    $\begingroup$ @ttnphns - Well, I learned something! Of my sources, only Lehmann & Romano (which of course I haven't memorized) describes the Wilcoxon as testing symmetry, the others (Hollander & Wolfe, Randles & Wolfe) describe it as assuming symmetry and testing for a shift. OTOH, I believe the Mann-Whitney's HO really is $F=G$, not just the complement of "stochastic dominance", because the dist'n of the test statistic is not distribution-free if $F \neq G$; consider testing a N(0,1) vs [0.01 of N(-5,1) and 0.99 of N(5,1)]; no stochastic dominance, but the dist'n of U is not the same as N(0,1) vs N(0,1). $\endgroup$
    – jbowman
    Commented Nov 15, 2013 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.