I have some confusion related to Kruskal wallis test. I have an example lets say

X=[2 2 35 10 9 8 11 12];
Y=[1 1 1 2 2 2 2 2];

Y is the group variable

Now when I ran the kruskalwallis test

p = kruskalwallis(X,Y,'off')

I got p values of around 0.4. I was assuming the Kruskal wallis test takes the median. So it should have been robust when I added an outlier with value 35 in the third position. Why isn't it robust to that. Is it because I have very few samples. Can anyone explain?

  • $\begingroup$ I get a p-value of 0.0046 or 0.0030 with ties. Are you sure you're doing this test correctly? $\endgroup$
    – dimitriy
    Commented Apr 2, 2013 at 23:52
  • $\begingroup$ I got very different values with the outlier (chi-square = 0.38, df = 1, p = 0.54) and without (chi-square = 16.25, df = 1, p = 0.00005) running kruskal.test from the stats package in R; very similar with wilcox.test $\endgroup$
    – Peter Flom
    Commented Apr 3, 2013 at 0:17
  • $\begingroup$ What package? Is that matlab? $\endgroup$
    – Glen_b
    Commented Apr 3, 2013 at 0:21
  • $\begingroup$ yeah I am using matlab $\endgroup$
    – user34790
    Commented Apr 3, 2013 at 0:27
  • $\begingroup$ @DimitriyV.Masterov. Yeah I am sure. How come you got 0.0046 abd 0.0030. I am using matlab. Are u using matlab? $\endgroup$
    – user34790
    Commented Apr 3, 2013 at 0:28

2 Answers 2


If Y is meant to be a grouping variable, the p-value in R is around 0.45

> kruskal.test(x~y)

    Kruskal-Wallis rank sum test

data:  x by y 
Kruskal-Wallis chi-squared = 0.5622, df = 1, p-value = 0.4534

But it makes no difference whether that 35 is set to 13 or 35 or 1300 - the p-value is exactly the same. It is clearly robust to outliers.

With continuity correction, the p-value is somewhat higher.


Here's an illustration of just how the Kruskal-Wallis p-value responds as you move the third observation around - that is, this is an empirical influence curve for the p-value as x[3] is moved (takes the various values of delta).

Kruskal-Wallis p-value as x[3] changes

We see that the Kruskal-Wallis is highly insensitive to all but a small range of values for x[3] (it is constant to the left of $[1,2]$ and constant to the right of it). It's really insensitive.

The grey line is the p-value with x[3] omitted. As you see, no value for x[3] will allow the Kruskal-Wallis to attain that p-value, though making x[3]=2 comes closest.

I was assuming the Kruskal wallis test takes the median.

It's a rank-based ANOVA. It doesn't actually 'use' the median for anything.

The measure of location-shift that corresponds to the Wilcoxon-Mann-Whitney (and hence to the Kruskal-Wallis) is the median of pairwise differences between the samples.

> median(outer(x[y==1],x[y==2],"-"))
[1] -7


> wilcox.test(x~y,conf.int=TRUE)

    Wilcoxon rank sum test with continuity correction

data:  x by y 
W = 5, p-value = 0.5486
alternative hypothesis: true location shift is not equal to 0 
95 percent confidence interval:
 -10   5 
sample estimates:
difference in location 
             -6.999992    #<-------------------------------

(I'm not sure why it doesn't have better accuracy there)

If you change the 35 to 13 or 1300, you get the same estimate of shift.

If you add a whole new observation - if your original data in the first group was just (2, 2), then adding an additional observation changes the p-value. (This would be the case even if the median was the estimate of location shift.)

  • $\begingroup$ I am still getting 0.4 in matlab. I have followed the syntax. Can anyone confirm in matlab? $\endgroup$
    – user34790
    Commented Apr 3, 2013 at 1:10
  • $\begingroup$ It may just be using a different approximation. The exact p-value is 29/56 = 0.5179. (Note that there are only 56 possible arrangements of 8 objects into two groups of 3 and 5 objects respectively.) $\endgroup$
    – Glen_b
    Commented Apr 3, 2013 at 1:24

You have the MATLAB syntax wrong, I think, or else I am testing the wrong hypothesis. If you have 2 samples of 8 observation each, then MATLAB expects the data to be in columns, so try p=kruskalwallis([X',Y']). This gives me a p-value of 0.0030, as in Stata.

Here's my MATLAB output for the first case:

>> X

X =

     2     2    35    10     9     8    11    12

>> Y

Y =

     1     1     1     2     2     2     2     2

>> [p,table]=kruskalwallis([X',Y'])

p =


table = 

    'Source'     'SS'          'df'    'MS'          'Chi-sq'    'Prob>Chi-sq'
    'Columns'    [182.2500]    [ 1]    [182.2500]    [8.8185]    [     0.0030]
    'Error'      [127.7500]    [14]    [  9.1250]          []               []
    'Total'      [     310]    [15]            []          []               []

If you meant for Y to be group variable, you get:

>> p=kruskalwallis(X',Y')

p =


>> p=kruskalwallis(X,Y)

p =



I am also very curious about this test, so I bootstrapped the adjusted p-value from the KW test. The distribution is shown below, with the observed p-value denoted by the dashed line. There's a lot of mass near 0, but also some near the observed p-value and near 1. Not quite sure what to make of that.

enter image description here

  • $\begingroup$ I have tried with the columns as well. It is still the same $\endgroup$
    – user34790
    Commented Apr 3, 2013 at 1:12
  • $\begingroup$ I am not sure what's going wrong then. $\endgroup$
    – dimitriy
    Commented Apr 3, 2013 at 1:18
  • $\begingroup$ That was my question. If I use Y as a group variable why do I get p value of 0.45 when I add an outlier. It means the test is not working properly. $\endgroup$
    – user34790
    Commented Apr 3, 2013 at 1:53
  • 2
    $\begingroup$ Your assertion that 'the test is not working properly' is mistaken. When you add an observation to a sample, you totally change the conditions; it doesn't matter where you add it, you'll change the p-value in some way, and with such tiny samples, it can change a lot. I showed you that when you move an observation instead, then as long as it was already outside the range of rest of the combined sample, the K-W doesn't respond to it at all as you move it further from the data. The K-W seems to be behaving just as it should. $\endgroup$
    – Glen_b
    Commented Apr 3, 2013 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.