# Confusion related to kruskal wallis test

I have some confusion related to Kruskal wallis test. I have an example lets say

X=[2 2 35 10 9 8 11 12];
Y=[1 1 1 2 2 2 2 2];


Y is the group variable

Now when I ran the kruskalwallis test

p = kruskalwallis(X,Y,'off')


I got p values of around 0.4. I was assuming the Kruskal wallis test takes the median. So it should have been robust when I added an outlier with value 35 in the third position. Why isn't it robust to that. Is it because I have very few samples. Can anyone explain?

• I get a p-value of 0.0046 or 0.0030 with ties. Are you sure you're doing this test correctly? – Dimitriy V. Masterov Apr 2 '13 at 23:52
• I got very different values with the outlier (chi-square = 0.38, df = 1, p = 0.54) and without (chi-square = 16.25, df = 1, p = 0.00005) running kruskal.test from the stats package in R; very similar with wilcox.test – Peter Flom - Reinstate Monica Apr 3 '13 at 0:17
• What package? Is that matlab? – Glen_b -Reinstate Monica Apr 3 '13 at 0:21
• yeah I am using matlab – user34790 Apr 3 '13 at 0:27
• @DimitriyV.Masterov. Yeah I am sure. How come you got 0.0046 abd 0.0030. I am using matlab. Are u using matlab? – user34790 Apr 3 '13 at 0:28

If Y is meant to be a grouping variable, the p-value in R is around 0.45

> kruskal.test(x~y)

Kruskal-Wallis rank sum test

data:  x by y
Kruskal-Wallis chi-squared = 0.5622, df = 1, p-value = 0.4534


But it makes no difference whether that 35 is set to 13 or 35 or 1300 - the p-value is exactly the same. It is clearly robust to outliers.

With continuity correction, the p-value is somewhat higher.

Edit:

Here's an illustration of just how the Kruskal-Wallis p-value responds as you move the third observation around - that is, this is an empirical influence curve for the p-value as x[3] is moved (takes the various values of delta).

We see that the Kruskal-Wallis is highly insensitive to all but a small range of values for x[3] (it is constant to the left of $[1,2]$ and constant to the right of it). It's really insensitive.

The grey line is the p-value with x[3] omitted. As you see, no value for x[3] will allow the Kruskal-Wallis to attain that p-value, though making x[3]=2 comes closest.

I was assuming the Kruskal wallis test takes the median.

It's a rank-based ANOVA. It doesn't actually 'use' the median for anything.

The measure of location-shift that corresponds to the Wilcoxon-Mann-Whitney (and hence to the Kruskal-Wallis) is the median of pairwise differences between the samples.

> median(outer(x[y==1],x[y==2],"-"))
[1] -7


Compare:

> wilcox.test(x~y,conf.int=TRUE)

Wilcoxon rank sum test with continuity correction

data:  x by y
W = 5, p-value = 0.5486
alternative hypothesis: true location shift is not equal to 0
95 percent confidence interval:
-10   5
sample estimates:
difference in location
-6.999992    #<-------------------------------


(I'm not sure why it doesn't have better accuracy there)

If you change the 35 to 13 or 1300, you get the same estimate of shift.

If you add a whole new observation - if your original data in the first group was just (2, 2), then adding an additional observation changes the p-value. (This would be the case even if the median was the estimate of location shift.)

• I am still getting 0.4 in matlab. I have followed the syntax. Can anyone confirm in matlab? – user34790 Apr 3 '13 at 1:10
• It may just be using a different approximation. The exact p-value is 29/56 = 0.5179. (Note that there are only 56 possible arrangements of 8 objects into two groups of 3 and 5 objects respectively.) – Glen_b -Reinstate Monica Apr 3 '13 at 1:24

You have the MATLAB syntax wrong, I think, or else I am testing the wrong hypothesis. If you have 2 samples of 8 observation each, then MATLAB expects the data to be in columns, so try p=kruskalwallis([X',Y']). This gives me a p-value of 0.0030, as in Stata.

Here's my MATLAB output for the first case:

>> X

X =

2     2    35    10     9     8    11    12

>> Y

Y =

1     1     1     2     2     2     2     2

>> [p,table]=kruskalwallis([X',Y'])

p =

0.0030

table =

'Source'     'SS'          'df'    'MS'          'Chi-sq'    'Prob>Chi-sq'
'Columns'    [182.2500]    [ 1]    [182.2500]    [8.8185]    [     0.0030]
'Error'      [127.7500]    [14]    [  9.1250]          []               []
'Total'      [     310]    [15]            []          []               []


If you meant for Y to be group variable, you get:

>> p=kruskalwallis(X',Y')

p =

0.4534

>> p=kruskalwallis(X,Y)

p =

0.4534


Update:

I am also very curious about this test, so I bootstrapped the adjusted p-value from the KW test. The distribution is shown below, with the observed p-value denoted by the dashed line. There's a lot of mass near 0, but also some near the observed p-value and near 1. Not quite sure what to make of that.

• I have tried with the columns as well. It is still the same – user34790 Apr 3 '13 at 1:12
• I am not sure what's going wrong then. – Dimitriy V. Masterov Apr 3 '13 at 1:18
• That was my question. If I use Y as a group variable why do I get p value of 0.45 when I add an outlier. It means the test is not working properly. – user34790 Apr 3 '13 at 1:53
• Your assertion that 'the test is not working properly' is mistaken. When you add an observation to a sample, you totally change the conditions; it doesn't matter where you add it, you'll change the p-value in some way, and with such tiny samples, it can change a lot. I showed you that when you move an observation instead, then as long as it was already outside the range of rest of the combined sample, the K-W doesn't respond to it at all as you move it further from the data. The K-W seems to be behaving just as it should. – Glen_b -Reinstate Monica Apr 3 '13 at 7:19