# Using Kruskal–Wallis test on large data sets

My research involves a huge number of data points (millions). I have one independent variable with 3 groups (patent organizational type (University/Industry/Collaborative Patent)) and 4 ordinal independent variables (Citations (Forward and Backward)/family size/claims).

In order to check that there is a difference between the three types of organizations, I used Kruskal–Wallis (a nonparametric test) with all the three groups of patent organizational type. My questions are:

1. Is it okay to use Kruskal–Wallis with huge data (N=Millions)?

And if I have inequivalent N with the 3 groups, for example examining FW Citation:

• University Patents = 14000
• Industry Patents = 819161
• Collaborative Patents = 997

2. Is there any issue with using Kruskal Wallis in this case?

• What are your concerns? // My concerns are: 1) It might take a long time 2) The p-values might catch differences that are not practically significant. For the second concern, that is a feature, not a bug, of hypothesis testing.
– Dave
Jun 21, 2021 at 20:41
• Both of David's concerns can be addressed with an appropriately-chosen resampling method. A tricky thing about Kruskall-Wallis, and similar stochastic dominance tests, is determining what the smallest meaningful effect size would be in terms of a test score. Jun 22, 2021 at 3:34

Comment:

As the Kruskal-Wallis test is implemented in R, only the second of @Dave's concerns arises. (Running time on my elderly computer is less than 5 seconds, consistent with time required to sort data.)

set.seed(2021)
x1 = rgamma( 14000, 5, .111)
x2 = rgamma(220000, 5, .112)
x3 = rgamma(  1000, 5, .113)
median(x1); median(x2); median(x3)
[1] 41.98856
[1] 41.36085
[1] 41.60076
x = c(x1,x2,x3)
g = rep(1:3, c(14000,220000,1000))
boxplot(x~g); abline(h=40, col="green")


The K-W test is significant at the 6% level (for fictitious data from the seed shown).

kruskal.test(x~g)

Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 5.7846, df = 2,
p-value = 0.05545


Just looking at the x1 and x2, one finds a significant difference (of a size that may not be of practical importance).

wilcox.test(x1,x2)

Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 1557800000, p-value = 0.02178
alternative hypothesis: true location shift is not equal to 0