Mann Whitney Test: Clearing Up Confusion

Question

I have been reading many statistical websites stating that the Mann Whitney test is a test of medians. However, I believe that this is not really true? It is a test of the difference in the ranks. The Mann-Whitney test only tests for a difference in medians when you assume that the only difference in the distributions of the two samples is the location, and not the scale or shape, of the distribution, which is very often too strong an assumption. Additionally, if one does make this assumption, then I believe it's also fair to say that the Mann Whitney test compares the difference in means as well.

I have a few questions that relate to this:

1. I am a bit confused why, often in research papers, the medians are reported when stating the results of a Mann Whitney test. It seems that it's only under rare occasions when the Mann Whitney test is actually able to compare the medians. Additionally, do researchers even check to see whether distributions are equal in the first place before saying that the test is a comparison of medians? If not, then it seems a bit unsound to report the medians.

2. Is the Mann Whitney test comparing the distribution of the ranks for two groups?

3. I am also a bit confused about what is stated here. Reporting Results of Mann-Whitney U Test - Means vs Medians . The first post states that "the location-difference measure that the Mann-Whitney 'sees' is neither difference in means nor difference in medians – it's the median of cross-group pairwise differences (the between samples quantity is the relevant estimate of the corresponding measure between populations)". How exactly does "the median of cross-group pairwise differences" relate to ranks.

Answer 1

1. It's quite reasonable to report the medians when doing a Mann-Whitney test. The median is perfectly good location summary. The problem is describing the test as a test for difference in median. I think this is the fault of statistics textbooks – it's a very common mistake in them. I don't know of the original source of this idea, though I have spent quite a lot of time looking, over the years. My guess is that someone recommended doing a Mann-Whitney test and also quoting medians, and it got misunderstood as saying the medians were what was being tested. But that's just a guess.

2. Yes. It's equivalent to the Wilcoxon rank-sum test, which is basically a t-test on the ranks. It depends on the ranks, and nothing else.

3. Suppose you fix one sample (call it $X$) and shift the other sample (call it $Y$) up or down (by adding or subtracting the same number $\delta$ from each observation) until the Mann-Whitney test statistic has exactly its null value. At that point, the number of pairs with $X_i> Y_j+\delta$ will be the same as the number of pairs with $X_i< Y_j+\delta$, because the test is based on the difference of those two numbers. That is, if you look at $X_i-(Y_j+\delta)$, over all $(i,j)$ pairs, half will be positive and half will be negative: the median pairwise difference will be zero. The Hodges-Lehmann estimator corresponding to a rank test is the amount you have to slide the data to make the test perfectly null, so $\delta$.

4. It's slightly misleading to say the Mann-Whitney is a test for the median pairwise difference. It is, but not in the same way that the t-test is a test for the mean or Mood's median test is a test for the median. If $X$ has a higher mean than $Y$ and $Y$ has a higher mean than $Z$, then you know $X$ has a higher mean than $Z$. If $X$ has a higher median than $Y$ and $Y$ has a higher median than $Z$, then you know $X$ has a higher median than $Z$. But if the median pairwise difference for $X$ and $Y$ is positive and the median pairwise difference for $Y$ and $Z$ is positive, this does not guarantee that the median pairwise difference for $X$ and $Z$ is positive. The Mann-Whitney test is not transitive, and it is not consistent with any ordering on all possible distributions – or even with any ordering on quite small sets of distributions.

Answer 2

The Mann-Whitney U test (MWUt from now on) does not compare medians, nor compares distributions. Why it is used this way in many published papers, presented this way in many textbooks or web sites is beyond me, and is a sad testimony on the teaching of statistics. The MWUt is simply a test of stochastic superiority: given populations A and B, when one takes a random single value from A and another random single value from B, does one population (A or B) more often than not have greater values. The null is P(A>B)=P(B>A)=.5. Trying to use it for comparing medians imposes drastic conditions on the population's probability distributions (both symmetrical, or both exactly identical -the same!- just shifted), which are untestable, and in practice never realized (because, in practice, we only have the samples, and your samples will never be both exactly symmetric, nor exactly the same shape, just shifted). This lack of symmetry and/or similarity will influence the U (or T) statistic. A good source which explains this simply and clearly is https://www.graphpad.com/guides/prism/latest/statistics/stat_nonparametric_tests_dont_compa.htm I quote this page: "The Mann-Whitney test compares the mean ranks -- it does not compare medians and does not compare distributions. More generally, the P value answers this question: What is the chance that a randomly selected value from the population with the larger mean rank is greater than a randomly selected value from the other population?"

Note that even if one wanted to interpret the MWUt as a test of equality of distributions, the alternate hypothesis would be of no value: the alternate would be that either the 2 populations are not symmetrical (maybe only 1 is, maybe none are), or are not the same (i.e. have somehow different pdf's): what do you do with this? Do you pick A or B? The alternate, if accepted, is not interpretable. But stochastic superiority is usable and interpretable: I would pick the population which is more often larger than the other, because, in some sense, it is "better" (yes, many caveats about understanding why A achieved superiority, but the logic is not much different from picking the population with the larger mean, or median, etc...). Btw, Mann & Whitney titled their original paper "On a test of whether 1 of 2 random variables is stochastically larger than the other", not "On a test of the medians"...

Answer 3

The best way to see that the Mann-Whitney test does not compare medians is to see two data sets with identical medians (horizontal lines) but a small p-value (0.03) from the Mann-Whitney test:

Source