Can I conduct a Kruskal Wallis test on 4 groups with very different sample sizes? I have a paper under review where I conducted a Kruskal Wallis test in SPSS to compare 4 groups. Each group contains payments made by companies over one year, and group 1 received substantially more payments than the other groups.
Group 1 - 18,190
Group 2 - 1,370
Group 3 - 990
Group 4 - 311
A reviewer suggested that for group 1, I should use a random 10% of the sample, however, I cannot find references to support doing this. I think removing 90% of the Group 1 sample would ignore most of the data.
Is anyone able to advise whether it is acceptable to run a KW on 4 very different groups?
 A: This is related to a disturbing but very common misunderstanding.
Tests for one-way type comparisons (one-sample location tests) are designed for different sample sizes. There's no harm in the sample sizes being different, as long as the assumptions are reasonable (beware, also, choosing your tests based on what you find in the data; this can sometimes cause the sort of problem you're trying to avoid).
If you're designing an experiment or some such, it's best to allocate the total sample size equally to each group but if you have sample sizes that are unequal, that preference is not a consideration.

*

*you lose power by reducing sample size in the large groups, for no obvious gain anywhere else


*if you're throwing away information at random, you also get the very strange property that two people using exactly the same test on exactly the same original data may come to different conclusions. This is not my idea of a desirable property.


*if robustness to unequal spreads is an issue (which can matter if it occurs under H0), you'd be better modifying your choice of test than throwing away data. You cannot typically judge whether this is an issue from the data (since the null is likely false and spread that changes as the location changes could easily produce differing spreads under H1). Beware taking drastic action to compensate for this unnecessarily; this may be considerably less likely to be a problem than many people seem to think.

advise whether it is acceptable

Acceptable to whom? There's not necessarily any statistical issue, but that doesn't mean a colleague (e.g. a reviewer) won't cause you problems; they may even point to papers in your application area (whatever it might be) that make the same sort of claim.
