1
$\begingroup$

My research involves a huge number of data points (millions). I have one independent variable with 3 groups (patent organizational type (University/Industry/Collaborative Patent)) and 4 ordinal independent variables (Citations (Forward and Backward)/family size/claims).

In order to check that there is a difference between the three types of organizations, I used Kruskal–Wallis (a nonparametric test) with all the three groups of patent organizational type. My questions are:

1. Is it okay to use Kruskal–Wallis with huge data (N=Millions)?

And if I have inequivalent N with the 3 groups, for example examining FW Citation:

  • University Patents = 14000
  • Industry Patents = 819161
  • Collaborative Patents = 997

2. Is there any issue with using Kruskal Wallis in this case?

$\endgroup$
2
  • 5
    $\begingroup$ What are your concerns? // My concerns are: 1) It might take a long time 2) The p-values might catch differences that are not practically significant. For the second concern, that is a feature, not a bug, of hypothesis testing. $\endgroup$
    – Dave
    Jun 21, 2021 at 20:41
  • $\begingroup$ Both of David's concerns can be addressed with an appropriately-chosen resampling method. A tricky thing about Kruskall-Wallis, and similar stochastic dominance tests, is determining what the smallest meaningful effect size would be in terms of a test score. $\endgroup$
    – Galen
    Jun 22, 2021 at 3:34

1 Answer 1

1
$\begingroup$

Comment:

As the Kruskal-Wallis test is implemented in R, only the second of @Dave's concerns arises. (Running time on my elderly computer is less than 5 seconds, consistent with time required to sort data.)

set.seed(2021)
x1 = rgamma( 14000, 5, .111)
x2 = rgamma(220000, 5, .112)
x3 = rgamma(  1000, 5, .113)
median(x1); median(x2); median(x3)
[1] 41.98856
[1] 41.36085
[1] 41.60076
x = c(x1,x2,x3)
g = rep(1:3, c(14000,220000,1000))
boxplot(x~g); abline(h=40, col="green")

enter image description here

The K-W test is significant at the 6% level (for fictitious data from the seed shown).

kruskal.test(x~g)

         Kruskal-Wallis rank sum test

data:  x by g
Kruskal-Wallis chi-squared = 5.7846, df = 2, 
  p-value = 0.05545

Just looking at the x1 and x2, one finds a significant difference (of a size that may not be of practical importance).

wilcox.test(x1,x2)

        Wilcoxon rank sum test with continuity correction

data:  x1 and x2
W = 1557800000, p-value = 0.02178
alternative hypothesis: true location shift is not equal to 0
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.