# Reconciling seemingly different null hypotheses for Mann-Whitney test

The Mann-Whitney test ( Rank sum, Wilcoxon-Mann-Whitney ) is according to some authors testing

H0: the populations from which the 2 samples were drawn are identical in every respect

e.g. see Hoel Introduction to Mathematical Statistics, 5th Edition, p342 and also on this site Mann-Whitney test interpretation

Other times it is described as

H0: one variable is stochastically larger than the other

indeed, that is the essence of the title of the Mann-Whitney paper of 1947 in Annals of Mathematical Statistics. and also on this site Interpretation of Wilcoxon Rank Sum test results

One can perform a quick numerical experiment (R)

set.seed(123)

n <- 10

x1 <- rnorm(n, mean=0, sd=1)
x2 <- rnorm(n, mean=0, sd=3)

print( wilcox.test(x1,x2))


yielding the output

    Wilcoxon rank sum test

data:  x1 and x2
W = 39, p-value = 0.4359
alternative hypothesis: true location shift is not equal to 0


Note x2 has 3 times the sd of x1 so the populations are not identical. Yet, the test, if I am interpreting the result correctly, claims no evidence against H0.

If the first view of H0 holds, why doesn't the test reject the hypothesis the populations are identical.

These 2 views of H0 do not seem equivalent. How are they reconciled?

• From the R documentation: if paired is TRUE, a Wilcoxon signed rank test of the null that the distribution of x (in the one sample case) or of x - y (in the paired two sample case) is symmetric about mu is performed. Otherwise, the null hypothesis is that the distributions of x and y differ by a location shift of mu and the alternative is that they differ by some other location shift (and the one-sided alternative "greater" is that x is shifted to the right of y). – user2974951 Aug 23 '19 at 10:07
• – Sal Mangiafico Aug 23 '19 at 16:50
• "H0: one population (group) is NOT stochastically larger than the other" - this is the null hypothesis (your formulation is the alternative hypothesis, and not "variable", - group). – ttnphns Aug 23 '19 at 17:38
• @ttnphns , "You may take one group, a symmetric continuous distribution...": that's exactly the OP's question. If the test discovers only cases of stochastic equality, why is the H0 often described as: The two distributions are identical. ? – Sal Mangiafico Aug 23 '19 at 18:20
• The Question is unclear. Something is misunderstood. @SalMangiafico is entitled to his several opinions as to what. – BruceET Aug 23 '19 at 18:44

This is a misunderstanding:

In the first link, it is specifically stated "$$H_0 :$$ The two population distributions are identical" and that point of view is consistently taken throughout.

In the second link, @Glen_b says that the test "considers whether" one variable is stochastically greater than the other. That is never said to be the null hypothesis, and it isn't. It is a statement of the alternative hypothesis. (In my opinion, the main point of that Answer is to explain that the Mann-Whitney-Wilcoxon does not simply test whether medians are equal.)

Addendum per additional information from OP. There are a couple of reasons you don't find a significant difference between two samples of size $$n = 10$$ from $$\mathsf{Norm}(\mu=0, \sigma=1)$$ and $$\mathsf{Norm}(\mu=0, \sigma=3),$$ respectively.

(1) The two population distributions do not have the same shape (variances differ).

(2) The difference between these two distributions is not their location (both centered at 0).

(3) Sample sizes are small.

(4) Data are normal, so tests based on normal theory will be more powerful.

As to (1) and (4): A (normal-theory) F test finds a significant difference in variances.

set.seed(123);  x1=rnorm(10,0,1);  x2=rnorm(10,0,3)
var.test(x1, x2)

F test to compare two variances

data:  x1 and x2
F = 0.0938, num df = 9, denom df = 9, p-value =
0.001628
alternative hypothesis:
true ratio of variances is not equal to 1
95 percent confidence interval:
0.02329852 0.37763707
sample estimates:
ratio of variances
0.09379971


As to (2) and (4): For normal data with differing means, both a (normal-theory) two-sample t test and a two-sample nonparametric Wilcoxon (signed-rank) test find highly significant differences in location.) For brevity here, I have used $-notation to show only the P-values.) Because the difference in location is substantial, compared with variability, significant differences are found in spite of small sample sizes. set.seed(123); y1 = rnorm(10,0,1); y2 = rnorm(10,3,1) t.test(y1, y2)$$p.val [1] 1.526441e-06 wilcox.test(y1, y2)$$p.val [1] 4.330035e-05  As to (3), sample sizes are too small to detect a smaller difference in centers. set.seed(123); y1 = rnorm(10,0,1); y2 = rnorm(10,.5,1) wilcox.test(y1, y2)$p.val
[1] 0.1431401


However, for substantially larger sample sizes ($$n=50$$), the Wilcoxon SR test is able to detect even the small shift.

set.seed(123); y1 = rnorm(50,0,1); y2 = rnorm(50,.5,1)
wilcox.test(y1, y2)

Wilcoxon rank sum test
with continuity correction

data:  y1 and y2
W = 781, p-value = 0.001239
alternative hypothesis:
true location shift is not equal to 0


Notches in the sides of the boxplots below are non-overlapping nonparametric confidence intervals, indicating a shift in location.

boxplot(y1, y2, notch=T, col="skyblue2", pch=20, names=T)


Note: Because data are normal, the last two tests could have been t tests. You can do those if you like. But I omitted them because our main topic here is Wilcoxon SR tests. You might try Wilcoxon tests for (non-normal) uniform data (with a location shift of 0.5):

set.seed(123); u1 = runif(10,0,1); u2 = runif(10,.5,1.5)


Addendum per comment: Here is a test for unequal variances between the two small normal samples in your Question. The null hypothesis #H_0: \sigma_1^2 = \sigma_2^2$is rejected at the 5% level. The Mann-Whitney-Wilcoxon 2-sample nonparametric test does not reject because the difference is in 'shape', not 'location'. set.seed(123) n <- 10 x1 <- rnorm(n, mean=0, sd=1) x2 <- rnorm(n, mean=0, sd=3) var.test(x1, x2) F test to compare two variances data: x1 and x2 F = 0.0938, num df = 9, denom df = 9, p-value = 0.001628 alternative hypothesis: true ratio of variances is not equal to 1 95 percent confidence interval: 0.02329852 0.37763707 sample estimates: ratio of variances 0.09379971 wilcox.test(x1,x2)$p.val
[1] 0.4358722


For slightly larger samples, neither 2-sample Wilcoxon nor 2-sample t test rejects.

set.seed(123)
n <- 30
x1 <- rnorm(n, mean=0, sd=1)
x2 <- rnorm(n, mean=0, sd=3)
wilcox.test(x1, x2)$$p.val [1] 0.3353956 t.test(x1, x2)$$p.val
[1] 0.2434323

• thanks that is helpful (+1). I am then confused about the test in R, it's docs say "Performs one- and two-sample Wilcoxon tests on vectors of data; the latter is also known as ‘Mann-Whitney’ test." I am doing the latter and the test should then test with the first H0, but the example I provided shows it does not (unless somehow it's not powerful enough to detect that difference?) – PM. Aug 23 '19 at 16:34
• The R procedure wilcox.test does a one-sample Wilcoxon signed-rank test if one vector of data is included as an argument. The test does a two-sample Mann-Whitney-Wilcoxon test if two data vectors are included (or an alternate 'equation' method is used). The Wilcoxon rank sum test and the Mann-Whitney test were originally published separately; it was subsequently discovered that the two tests do the same thing. // If you show or describe your data and objective, perhaps someone can say whether a MWW test is appropriate. – BruceET Aug 23 '19 at 16:45
• I'm extremely grateful for your (+anyone's) input. Let's say my data is as in the R snippet above. The variance differs. If MW does test for identical distributions, I don't understand why it doesn't indeed detect the data are drawn from different distributions and supply a p-value that allows me to reject H0. (As a courtesy note, I must leave the site for a while now.) – PM. Aug 23 '19 at 17:00
• @BruceET If it were possible to give more upvotes for this I would. It's very helpful to supply such an informative answer. But, your (1) captures the essence of the problem, the populations have different shapes. To me this means they are not identical? Is it incorrect to think this? Hence H0: f(x) = g(x) is false. But MW fails to detect this. Perhaps it's for reason (3). I should experiment with larger sample sizes but I feel there's something else fundamental at play here about MW H0. I appreciate I used normal data in the experiment, but that's just a convenience for illustration. – PM. Aug 24 '19 at 10:53
• A test may be designed for a specific kind of alternative; it needn't necessarily have power against every way in which the null+assumptions can be violated. – Glen_b Sep 23 '19 at 5:01

I don't know if this clears it up, but this is how Conover and Wilcox handle the hypotheses for the Mann-Whitney test.

Conover, 1999, Practical Nonparametric Statisitcs, 3rd.

Let F(x) and G(x) be the distribution functions corresponding to X and Y, respectively. Then the hypotheses may be stated as follows. H0: F(x)=G(x) for all x. H1: F(x)≠G(x) for some x.

... In many real situations any difference between distributions implies that P(X>Y) is no longer equal to 1/2. Therefore, H1: P(X>Y) ≠ P (X [less than] Y) is often used instead of the above.

Wilcox, 2017, Modern Statistics for the Social and Behavioral Sciences A Practical Introduction, 2nd

Let p be the probability that a randomly sampled observation from the first group is less than a randomly sampled observation in the second. H0: p = 0.5 (7.24)

... In a very real sense, a more accurate description of the Wilcoxon-Mann-Whitney test is that it provides a test of the hypothesis that the two distributions are identical. ... When the two distributions are identical, a correct estimate of the standard error [...] is being used. But otherwise, under general conditions, an incorrect estimate is being used, which results in practical concerns, in terms of both Type I errors and power, when using Equation (7.24) to test H0:p=0.5.

• Maybe, just maybe, I can understand all this with the statement "under general conditions, an incorrect estimate is being used, which results in practical concerns, in terms of both Type I errors and power". Thanks. – PM. Aug 24 '19 at 10:57
• One thing to think about is that these tests is that they are concerned with evaluating a population from which the data you have is a sample. It is the case that, for the data you have, the test will return a significant result only if sample p(x<y)≠0.5. But if you have a case where the populations are like your example, where P(X<Y)=0.5 but the distributions are not identical, the type I error will be inflated. That is, the test is reporting a significant p value greater than the nominal rate in this case. I'll post some R code in the following comment that exemplifies this. – Sal Mangiafico Aug 24 '19 at 13:55
• So I think this is just the way the test is. The null being tested is that two populations are identical, but it will report a significant result only when the samples reflect p(x<y)≠0.5. But sometimes this results when the populations exhibit P(X<Y)=0.5 just based on random sampling. – Sal Mangiafico Aug 24 '19 at 13:58
• Here, A and B have the same location, but different variances. n is the number of replications. m is the size of the sample. You can change this to e.g. 12 or 100. The type I error is reported. Compare to when A and B represent the same population. You can run here: rdrr.io/snippets . set.seed(sum(utf8ToInt("Hypotheses"))); A = rnorm(1000, 0, 1); B = rnorm(1000, 0, 4); n = 1000; m = 26; sig = rep(NA, n); for(i in 1:n){a = sample(A, m); b = sample(B, m); sig[i]=0; if(suppressWarnings(wilcox.test(a,b)\$p.value<=0.05)){sig[i]=1}}; sum(sig)/n – Sal Mangiafico Aug 24 '19 at 14:07
• thanks, I will take some time to ponder this. It is stimulating much thought. Thanks again. – PM. Aug 24 '19 at 20:25