I have seen that there exists tests which see if k samples share the same variance such as the Levene's test.

I was wondering if there is an equivalent test when means are wanting to be examined.

An non-parametric test would be needed here.

  • $\begingroup$ Most non-parametric tests are based on a stronger or slightly different null hypothesis, such none of the distributions is stochastically dominant. See en.wikipedia.org/wiki/…. $\endgroup$
    – whuber
    Feb 9, 2021 at 17:41
  • $\begingroup$ @whuber Does stochastic domaine imply difference in mean? I have seen people use the Mann Whitney U test for a difference in medians so does this not apply to the Kruskal-Wallis test? $\endgroup$
    – math111
    Feb 9, 2021 at 18:24
  • 2
    $\begingroup$ Yes, stochastic dominance implies a difference in mean--but the implication does not go the other way. $\endgroup$
    – whuber
    Feb 9, 2021 at 18:57
  • $\begingroup$ @whuber If I dont reject the null, do I conclude the samples are from the same distribution? The reason I ask is I also performed the Levene test and rejected the null implying a difference in variance. So how can the KW test tell us the distributors are the same. $\endgroup$
    – math111
    Feb 9, 2021 at 19:09
  • 1
    $\begingroup$ The null in the KW test does not assert all distributions are identical. $\endgroup$
    – whuber
    Feb 9, 2021 at 19:14

1 Answer 1


Valid conclusions from a Kruskal-Wallis test can be difficult when populations have markedly different shapes. Here we use three samples of size $n=100$ from $\mathsf{Beta}(.1,.1)$ with mean $\mu_1 = 1/2,$ $\mathsf{Beta}(5,1)$ with mean $\mu_2 = 5/6,$ and $\mathsf{Beta}(3,3)$ with mean $\mu_3 = 1/2.$

Densities of the three distributions are shown below:

enter image description here

Sampling in R:

x1 = rbeta(100, .1, .1)
x2 = rbeta(100, 5, 1)
x3 = rbeta(100, 3, 3)
x = c(x1,x2,x3);  gp=rep(1:3, each=100)

The three sample means are as follows:

mean(x1); mean(x2); mean(x3)
[1] 0.5259448
[1] 0.8361308
[1] 0.498663

hdr="Boxplots of Samples from BETA(.1,.1) [red], BETA(5.1) [green], BETA(3,3)"
boxplot(x ~ gp, col=c("red", "green", "blue"), main=hdr)

enter image description here

A Kruskal-Wallis test (not ordinarily recommended for comparing distributions of different shapes) shows a highly significant difference among the three samples.

kruskal.test(x ~ gp)

        Kruskal-Wallis rank sum test

data:  x by gp
Kruskal-Wallis chi-squared = 59.016, df = 2, p-value = 1.531e-13

Sometimes empirical CDF (ECDF) plots are used to judge visually whether one sample stochastically dominates another. A dominating sample will tend to plot to the right (hence below) samples it dominates. Here it seems clear that Sample 2 (green) dominates Sample 3 (blue), but not so clear whether Sample 2 dominates Sample 1 (red).

hdr="ECDFs of Samples from BETA(.1,.1) [red], 
     BETA(5.1) [green], BETA(3,3)"
plot(ecdf(x1), col="red", main=hdr)
 lines(ecdf(x2), col="green2")
 lines(ecdf(x3), col="blue")

enter image description here

Sometimes two-sample Wilcoxon rank sum tests are used ad hoc to determine differences is 'location' or 'domination'. Here are the P-values from two such tests, but I will leave the interpretation up to you. Ordinarily, one would want to see a P-value below about 1% or 2% (according to the Bonferroni method of avoiding 'false discovery') in order to declare differences.

[1] 0.06075133
[1] 2.750473e-24

Roughly estimated probabilities $P(X_2 > X_1)\approx 0.58$ and $P(X_2 > X_3)\approx 0.92.$

quantile(replicate(10^6, mean(sample(x2) > x1)), c(.025,.975))
 2.5% 97.5% 
 0.54  0.61 
quantile(replicate(10^6, mean(sample(x2) > x3)), c(.025,.975))
 2.5% 97.5% 
 0.88  0.95 
  • $\begingroup$ Thank you for your answer. To conclude, there are no tests that can compare means without the need of the same shape? $\endgroup$
    – math111
    Feb 10, 2021 at 10:33
  • $\begingroup$ No simple answer. Need to specify exact situation. Sometimes 'same shape' means same var also. Welch t test will compare means of 2 normal distributions with different variances. // Kolmogorov-Smirnov will compare samples to see if their populations have same shape. Might distinguish btw large enough samples from $\mathsf{Norm}(\mu=\sigma=1)$ and $\mathsf{Exp}(1).$ which has unit mean and SD: ks.test(rnorm(10,2,1),rexp(10))$p.val may not reject, but ks.test(rnorm(100,1,1),rexp(100))$p.val often does. // Can find LR test to distinguish btw 2 exp'l distn's w/ $\ne$ means. $\endgroup$
    – BruceET
    Feb 10, 2021 at 11:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.