# Performance of the Wilcoxon-Mann-Whitney test with large sample sizes (> 100,000) from medical data warehouses

Two large data sets of patients (set A: n = 100,000; set B: n = 700,000) are being compared with respect to their hospital length-of-stay (LOS). The variable LOS is reported as an integer in the data warehouse, typically between 2 and 7, so tens of thousands of patients in each group will have the same value for LOS. Does group A have a different LOS from group B? The variances of the LOS of the two groups are very different from one another; both have a skewed distribution.

In the medical literature one typically sees a Wilcoxon-Mann-Whitney test being used in comparing the LOS of the two groups and reports it as a test of the difference between medians. Usually due to unequal variance and sample size, such an approach does not conform to the so-called "pure shift model."

I want to avoid using this popular but flawed approach, but have two questions:

1. If I am using the W-M-W test as originally intended (testing the null hypothesis Prob(x < Y) = 0.5), will the unequal samples sizes (100,000 vs. 700,000) or the unequal variances invalidate the test result?

2. Is there a good test for comparing the median LOS of the two groups? It turns out that group A has a median LOS of 4 days while group B has a median LOS of 5 days, so a priori, given the large sample sizes, one might expect the two groups to significantly differ in their median LOS.

• With such huge sample sizes I would first be preoccupied about the independence assumption, there is bound to be some substructure, subgroups, changes with time, ... which covariables do you have? Consider some kind of regression modeling! – kjetil b halvorsen Feb 24 '17 at 8:31
• By "independence assumptions" are you asking whether the effect of LOS on readmission is in part explained by other covariables such as age, sex, etc? For time to readmission, a Cox proportional hazards model shows LOS is an independent predictor of the event. In a small subgroup of a certain type of readmission, multivariate logistic regression shows LOS not to be an independent predictor. Thank you - E Del Giacco – Eric Del Giacco Feb 28 '17 at 4:24

Your sample sizes are so large it would be surprising not to find differences on almost any reasonable measure of difference between the population distributions.

In the medical literature one typically sees a Wilcoxon-Mann-Whitney test being used in comparing the LOS of the two groups and reports it as a test of the difference between medians.

You already seem to clearly understand that it's not really testing that without additional assumptions.

Usually due to unequal variance and sample size, such an approach does not conform to the so-called "pure shift model."

You don't need a pure shift alternative for it to be a test of equality of medians; however that does make it considerably easier to interpret a rejection.

If I am using the W-M-W test as originally intended (testing the null hypothesis Prob(x < Y) = 0.5), will the unequal samples sizes (100,000 vs. 700,000) or the unequal variances invalidate the test result?

Unequal sample sizes will not be an issue. Unequal variances is not in any way a problem for the Wilcoxon-Mann-Whitney -- though it may be an issue if you want to use it to test equality of medians (in particular, if you want to insist that your alternatives may only be location-shifts). The discreteness is at least as much an issue for that as the variance.

There may be challenges to this sort of approach, but they don't really come from those directions.

Is there a good test for comparing the median LOS of the two groups?

Have you considered a permutation test with the test statistic the difference in sample medians? It will not be possible to compute the exact permutation distribution but it could be sampled to any desired accuracy.

There's also Mood's median test. It's perhaps a bit low-powered but that's probably not much of an issue with that sort of sample size.

I really don't see a great need for a test here though (you'll reject); what's likely to be a bit more interesting would be to give an interval for the difference in medians.

• Consider reviewing: Austin PC, et al. Health Services and Outcome Research Methodolgy. 3: 107-133, 2003 for a discussion of various approaches to length of stay modeling. – Todd D Feb 24 '17 at 7:12
• To Glen b: I am just a physician, not a statistician, so please elaborate what you mean by "an interval for the difference in medians." Thank you. – Eric Del Giacco Feb 28 '17 at 4:33
• To Todd: Thank you for the reference. I have asked my librarian to get me a copy. – Eric Del Giacco Feb 28 '17 at 4:34
• @EricDelGiacco You're interested in comparing medians; specifically I presume you want to infer something about the difference in population medians (speak up if that's not so!). Are you familiar with the notion of a confidence interval? – Glen_b -Reinstate Monica Feb 28 '17 at 4:39
• Incidentally, if your length of stay variable could be censored (e.g. if you can't observe everyone to the completion of their stay), you should probably be looking at survival analysis. – Glen_b -Reinstate Monica Feb 28 '17 at 4:40