I am finding myself having to perform non-parametric tests, primarily Mann-Whitney tests, on datasets which are not normally distributed.

I notice that the datasets are not homoscedastic - do these non-parametric tests, such as Mann-Whitney tests, require homoscedastic data?

  • 1
    $\begingroup$ If you do a 2-sample Mann-Whitney-Wilcoxon (signed rank test) for the purpose of determining whether two samples have the same median, then the two samples should be of the same shape (possibly differing by a shift in location). 'Same shape' implies 'same dispersion'. If the two samples have different dispersions (variances), then the M-M-W can be interpreted as a test whether one sample 'stochastically dominates' the other (tends to have consistently larger values than the other). But with heteroscedasticity it is not appropriate to talk just in terms of different variances. $\endgroup$ – BruceET Dec 27 '20 at 20:53

Simple shift. Consider two non-normal samples of the same shape, but with different medians. [Sampling and computations in R.]

x = rgamma(50, 5, 0.1)
y = rgamma(70, 5, 0.1) + 10
summary(x); length(x); sd(x)
    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  17.48   35.56   48.47   49.92   57.80  161.96 
[1] 50          # sample size
[1] 23.32434    # sample SD

summary(y); length(y); sd(y)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  22.84   39.90   56.45   60.10   66.46  173.68 
[1] 70
[1] 28.04877

stripchart(list(x,y), ylim = c(.5,2.5), pch=20)

enter image description here

The stripchart illustrates and the test confirms at the 5% level that the two samples have different locations. So we can say that the median 48.44 of Sample x is significantly different from the median 56.45 of sample y.

wilcox.test(x, y)

    Wilcoxon rank sum test with continuity correction

data:  x and y
W = 1345, p-value = 0.0313
alternative hypothesis: true location shift is not equal to 0

Different shapes. Although the Wilcoxon signed rank test was originally intended to test for a shift in location, resulting in a difference in medians, it can also be used to detect if one distribution 'dominates' another (tends to give larger values).

Consider the following two samples from beta distributions of different shapes. Medians are obviously different and variances are essentially the same.

y1 = rbeta(100, 2, 5)l  y2 = rbeta(100, 5,2)
median(y1); median(y2)
[1] 0.2749052
[1] 0.7259123
sd(y1); sd(y2)
[1] 0.1785754
[1] 0.1538957

Boxplots show that one distribution is right-skewed (bottom) and the other is left-skewed. Medians are different. Notches in the sides of the boxes show nonparametric confidence intervals calibrated so that if two of them don't overlap (as here) then the medians may be significantly different.

 boxplot(y1,y2, notch=T, horizontal=T, 
         col=c("wheat","skyblue2", pch=19))

enter image description here

The Wilcoxon signed rank test finds a significant difference between the population distribution from which the samples were drawn (P-value near 0), but the difference is not so simple as recognizing that the medians differ. [The output in R always mentions 'location shift' as the alternative hypothesis, but sometimes one needs to think carefully about the reason for rejection when the P-value is small.]


        Wilcoxon rank sum test with continuity correction

data:  y1 and y2
W = 581, p-value < 2.2e-16
alternative hypothesis: true location shift is not equal to 0

Here one could say that the second population 'dominates' the first. Roughly, this means that the second population tends to have larger values. There are several kinds ('orders') of stochastic dominance, and you can google for explanations that match your theoretical level. One method of judging stochastic dominance is that the empirical CDF (ECDF) plot of the dominating sample falls below and to the right of the ECDF plot of the other.

plot(ecdf(y1), col="brown", 
     main="ECDF plots: Blue dominates")
 lines(ecdf(y2), col="blue")

enter image description here

[Note: To make the ECDF of a sample of size $n$, sort observations from smallest to largest. The ECDF starts at 0 on the left at $0$ and jumps up by $1/n$ at each sorted value, reaching $1$ at the right. If there is a tie with $k$ observations at the same value, then the jump at that value is $k/n]$

Different dispersions. The Wilcoxon signed rank test may or may not detect a difference between two populations if their samples have markedly different variances. Below the one sample (larger median) clearly dominates the other and the Wilcoxon test rejects.

x1 = runif(100, 2,3)
x2 = runif(100, 2,3.5)
median(x1); median(x2)
[1] 2.468052
[1] 2.845346
sd(x1);  sd(x2)
[1] 0.2757188
[1] 0.44725

boxplot(x1,s2, notch=T, horizontal=T, 
        col=c("wheat","skyblue2"), pch=19))

enter image description here

    Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 3067, p-value = 2.337e-06
alternative hypothesis: true location shift is not equal to 0

plot(ecdf(x2), col="blue", 
     main="ECDF plots: Blue dominates")
 lines(ecdf(x1), col="brown")

enter image description here

However, if the two populations were $\mathsf{Beta}(2,2)$ and $\mathsf{Beta}(7,7),$ then one would not expect a two-sample Wilcoxon test to reject. Both distributions have median $\eta =1/2,$ so sample medians should not be significantly different.

curve(dbeta(x, 2,2), 0, 1, col="brown", lwd=2, ylab="PDF", xlab="W1", 
      main="Density of BETA(2,2)")
 abline(h=0, col="green2")
curve(dbeta(x, 7,7), 0, 1, col="blue", lwd=2, ylab="PDF", xlab="W1", 
      main="Density of BETA(7,7)")
abline(h=0, col="green2")

enter image description here

For the samples below from these two distributions, sample medians are about the same and sample interquartile ranges (IQRs) differ considerably (about 0.26 and 0.19, respectively).

w1 = rbeta(100, 2,2);  w2 = rbeta(100, 7,7)
median(w1); median(w2)
[1] 0.5360535
[1] 0.4734114
IQR(w1); IQR(w2)
[1] 0.2640769
[1] 0.192385
boxplot(w1,w2, notch=T, horizontal=T, 
        col=c("wheat","skyblue2"), pch=19)

enter image description here

The P-value of the Wilcoxon test is 0,1136 (not significant at 5%).

w = c(w1,w2)
g = rep(1:2, each=100)

        Wilcoxon rank sum test with continuity correction

data:  w by g
W = 5648, p-value = 0.1136
alternative hypothesis: 
 true location shift is not equal to 0

If the main issue is whether one population is more disperse than another, then a nonparametric test of dispersion would be more appropriate than a Wilcoxon signed rank test, which is a test of location.

One possible test whether the two samples have significantly different dispersions is a permutation test using the ratio of the two sample IQRs as the metric. The observed value of this metric for our two samples is $1.373.$ For the permutation test, we randomly permute the 200 observations between the two groups (100 in each) and compute the IQR-ratio again. After many iterations we get a good idea of the permutation distribution of the IQR ratio under the null hypothesis that the the two populations are equally disperse.

rto.obs = IQR(w1)/IQR(w2);  rto.obs
[1] 1.372648
m = 10^4;  rto.prm = numeric(m)
for(i in 1:m) {
 w.prm = sample(w)
 rto.prm[i] = IQR(w.prm[g==1])/IQR(w.prm[g==2])}
mean(rto.prm > rto.obs)
[1] 0.0244

The ratios from permutation are seldom as large as the observed ratio. So a one-sided test that the first sample is more disperse than the second rejects the null hypothesis at the the 3% level of significance.

hdr="Permutation Distribution of IQR Ratios"
hist(rto.prm, prob=T, col="skyblue2", main=hdr)
abline(v = rto.obs, col="red", lwd=2, lty="dashed")

enter image description here

Reference: One elementary paper on permutation tests is Eudey (2010).

  • $\begingroup$ It seems to me your comment answers the question but the answer does not. $\endgroup$ – Richard Hardy Dec 28 '20 at 6:56
  • $\begingroup$ @RichardHardy. To clarify: I considered my Ans to be a continuation of my very long Comment for the case where there is a true shift (even if slightly different SDs). Was going to illustrate different shapes, but got acceptance and upvote before I got to that, so I figured OP is mainly interested is the pure shift case. See here for case with different shapes. $\endgroup$ – BruceET Dec 28 '20 at 7:16
  • $\begingroup$ I see. I just thought the question was about heteroskedasticity but the answer did not discuss that. And just now I realized in addition that heteroskedasticity was probably meant across samples rather than within, an unsual use of the term. $\endgroup$ – Richard Hardy Dec 28 '20 at 7:29
  • 1
    $\begingroup$ I was interested in both cases @BruceET, but your answer was helpful to me in clearing my misunderstandings and allowing me to understand both in turn. I may have accepted the answer prematurely for the question, but I believe I realise now that my initial question wasn't really what I needed to know in the first place. $\endgroup$ – blammo69 Dec 28 '20 at 11:16
  • 2
    $\begingroup$ Thank you very much for the clarity and depth of your answer+edits, and a happy holidays to you too :). $\endgroup$ – blammo69 Dec 28 '20 at 21:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.