I've found exactly one source adressing this (and of course didn't save it). It said that in a Kruskal-Wallis this is a consequence of having a large sample with a lot of ties. Seen as I've got about 50,000 respondents and only an 11 point scale variable I'd say I qualify for both.
What that source didn't say however is how to treat this anomaly. At first I just treated is as if $p > 0.05$ and not significant. however when I loaded the wrong data in my post hoc analysis (the data with the negative H) a lot of the pairwise comparisons turned out to be significant. (even more so then some of the test where the H value had a $p < 0.001$)
So that made me wonder if I have to treat this negative H value differently. Should I just use a random subsample of data to see whether that has a significant H value or declare the test invalid and just see what happens with post hoc (the latter seems unlikely).
By the way my post hoc consists of a Bonferroni corrected Mann-Whitney U comparison.
-
$\begingroup$ Can you paste in the output you're wondering about? (BTW, you shouldn't use U post hoc after KW, try Dunn's test.) $\endgroup$– gung - Reinstate MonicaCommented Jun 28, 2016 at 13:52
-
1$\begingroup$ The definition formula has a ratio of two quantities both of which involve a sum of squared quantities. The problem comes, I suspect, from using a short-cut which replaces the denominator with its known value if there are no ties. $\endgroup$– mdeweyCommented Jun 28, 2016 at 14:00
-
$\begingroup$ I'm afraid I don't really have a traditional output from a statistical program. Because of tempory limitations I only have excel to work with. (I checked my calculations of course and it works on other data) So I basically constructed a kruskall-Wallis template from scratch. But going on an example this would be what SPSS would roughly give:<br> n group 1 9,943; mean rank 24,149<br> n group 2 10,091; mean rank 24,412<br> n group 3 9,223; mean rank 24,708<br> n group 4 8,584; mean rank 24,778<br> n group 5 8,164; mean rank 25,183<br> n group 6 3,281; mean rank 25,053<br> H=-2,2871 df=5 $\endgroup$– MennoCommented Jun 28, 2016 at 14:16
-
$\begingroup$ @mdewey That might indeed have been my mistake. Thanks for the help (you too @gung) $\endgroup$– MennoCommented Jun 28, 2016 at 14:28
-
$\begingroup$ The Mann-Whitney $U$ (rank sum) test is inappropriate as a post hoc test following rejection of Kruskal-Wallis because (a) it uses a different set of ranks (i.e. is effectively analyzing different data), and (b) it does not use the pooled variance implied by the null hypothesis of the Kruskal-Wallis test. This has been written about repeatedly on CV, so there's lots of places to learn about it. :) Also: Welcome, to CV! $\endgroup$– AlexisCommented Aug 21, 2018 at 14:03
1 Answer
The Kruskal-Wallis $H$ statistic is given by:
$$H=\frac{\frac{12\sum_{i=1}^{k}{n_{i}\left(\bar{R}_{i}-\bar{R}\right)^{2}}}{N\left(N+1\right)}}{1-\frac{\sum{T}}{N^{3}-N}}\text{, where:}$$
$k$ is the number of groups;
$N$ is the number of observations across all groups;
$n_{i}$ is the number of observations in the $i^{th}$ group;
$\bar{R}$ is the mean rank of all observations;
$\bar{R}_{i}$ is the rank sum of observations from the $i^{th}$ group (ranks are across observations from all groups); and
$T=t^{3}-t$ for each set of tied ranks, where $t$ is the number of ties in the set, and $\sum{T}$ is the sum of this quantity across all sets of tied ranks.
When there are no ties $T=0$, the denominator of $H$ simplifies to $1$.
For $N=50,000$ and a uniform distribution of ties across your eleven possible values the denominator of $H$ is approximately:
$$1-\frac{11\left(4545^3-45\right)}{50000^3-50000} \approx 0.9997$$
Assuming a highly skewed distribution of ties—say all but ten observations tied on a single value—the denominator of $H$ is approximately:
$$1-\frac{\left(49,990^3-49,990\right)}{50000^3-50000} \approx 0.0006$$
The most extreme case would be where all $N$ observations were tied on the same value, in which case the denominator of $H$ would simplify to $0$, and $H$ would thus be undefined.
Because the cubed term in $T$ can never be greater than $N^{3}$, I do not think it is possible to obtain a negative value of the denominator, and therefore not possible to obtain a negative value of $H$.
Conclusion:
- It is not possible to obtain a negative value of $H$ by adjusting for ties using Kruskal & Wallis formula for $H$ (Equation 1.2) and their adjustment for ties (Equation 1.3).
- Cubing a large $N$ might place one's software in the position of trying to calculate beyond its available precision, and numerical inconsistencies might thus result.
Kruskal, W. H. and Wallis, A. (1952). Use of ranks in one-criterion variance analysis. Journal of the American Statistical Association, 47(260):583–621.
