Performance of the Wilcoxon-Mann-Whitney test with large sample sizes (> 100,000) from medical data warehouses

Question

Two large data sets of patients (set A: n = 100,000; set B: n = 700,000) are being compared with respect to their hospital length-of-stay (LOS). The variable LOS is reported as an integer in the data warehouse, typically between 2 and 7, so tens of thousands of patients in each group will have the same value for LOS. Does group A have a different LOS from group B? The variances of the LOS of the two groups are very different from one another; both have a skewed distribution.

In the medical literature one typically sees a Wilcoxon-Mann-Whitney test being used in comparing the LOS of the two groups and reports it as a test of the difference between medians. Usually due to unequal variance and sample size, such an approach does not conform to the so-called "pure shift model."

I want to avoid using this popular but flawed approach, but have two questions:

1. If I am using the W-M-W test as originally intended (testing the null hypothesis Prob(x < Y) = 0.5), will the unequal samples sizes (100,000 vs. 700,000) or the unequal variances invalidate the test result?

2. Is there a good test for comparing the median LOS of the two groups? It turns out that group A has a median LOS of 4 days while group B has a median LOS of 5 days, so a priori, given the large sample sizes, one might expect the two groups to significantly differ in their median LOS.

Answer 1

Your sample sizes are so large it would be surprising not to find differences on almost any reasonable measure of difference between the population distributions.

In the medical literature one typically sees a Wilcoxon-Mann-Whitney test being used in comparing the LOS of the two groups and reports it as a test of the difference between medians.

You already seem to clearly understand that it's not really testing that without additional assumptions.

Usually due to unequal variance and sample size, such an approach does not conform to the so-called "pure shift model."

You don't need a pure shift alternative for it to be a test of equality of medians; however that does make it considerably easier to interpret a rejection.

If I am using the W-M-W test as originally intended (testing the null hypothesis Prob(x < Y) = 0.5), will the unequal samples sizes (100,000 vs. 700,000) or the unequal variances invalidate the test result?

Unequal sample sizes will not be an issue. Unequal variances is not in any way a problem for the Wilcoxon-Mann-Whitney -- though it may be an issue if you want to use it to test equality of medians (in particular, if you want to insist that your alternatives may only be location-shifts). The discreteness is at least as much an issue for that as the variance.

There may be challenges to this sort of approach, but they don't really come from those directions.

Is there a good test for comparing the median LOS of the two groups?

Have you considered a permutation test with the test statistic the difference in sample medians? It will not be possible to compute the exact permutation distribution but it could be sampled to any desired accuracy.

There's also Mood's median test. It's perhaps a bit low-powered but that's probably not much of an issue with that sort of sample size.

I really don't see a great need for a test here though (you'll reject); what's likely to be a bit more interesting would be to give an interval for the difference in medians.