5
$\begingroup$

The Mann-Whitney U Test can be used to test whether a randomly selected value from one distribution, $\ f_1(x)$, will be less than or greater than a randomly selected value from a second distribution, $\ f_2(x)$.

However when the two distributions are identical up to a simple translation, i.e. $\ f_1(x) = f_2(x + c)$, then a significant finding from this test implies that there exists a significant difference between the two populations' medians.

My question is, what's the best way to determine whether two populations are really identical up to translation? The most obvious thing that comes to mind is to translate the samples from the second distribution to bring it into alignment with the first, as determined by minimizing e.g. earth-mover distance, KL-divergence, or distance between medians, and then comparing the newly aligned distributions using the Kolmogorov-Smirnov test.

But I was wondering if there was a simpler one-step method, or some industry-standard methodology?

Improve this question
$\endgroup$
5
  • $\begingroup$ Why use such a complicated method? Subtracting the medians from each group will do the trick under the null hypothesis. However, I believe one wouldn't use a U test in general to test for arbitrary differences: it's not very powerful. To determine the "best" method we would need you to tell us what your alternative hypothesis really is. $\endgroup$
    – whuber ♦
    Jun 5, 2018 at 19:55
  • $\begingroup$ @whuber What I mostly care about is the core hypothesis being tested by the Mann-Whitney U Test, not the difference between the distribution medians. But if I can get that for free as well, then why not? And to get that, it's just a matter of knowing whether the translation assumption holds, which is why I'm asking about it. So you're saying that I should subtract the medians from both distributions, and then run a Kolmogorov-Smirnov test? $\endgroup$
    – jon_simon
    Jun 5, 2018 at 20:50
  • 2
    $\begingroup$ You could do that. Note that you cannot use the standard K-S distribution to compute p-values, so--to test significance--run a bootstrap or permutation calculation based on the K-S statistic. There are other ways to approach the problem, but given the power of the K-S test, this looks like an attractive, simple, and reasonably efficient solution. $\endgroup$
    – whuber ♦
    Jun 5, 2018 at 21:03
  • $\begingroup$ I don't understand why you can't compare the two distributions by just centering each by their mean and then looking at say, the difference of their CDFs? $\endgroup$
    – Alex R.
    Jun 5, 2018 at 21:53
  • 1
    $\begingroup$ The Kolmogorov-Smirnov statistic does look at the difference of their CDFs $\endgroup$
    – jon_simon
    Jun 5, 2018 at 22:35

1 Answer 1

Reset to default
1
$\begingroup$

You couldn't use a two-sample Kolmogorov-Smirnov test "as is" after you align the values in this way. The aligning would make the sample distributions more alike than they would be if the populations had the same location and you didn't align (i.e. the usual test will be overly conservative).

It will also not be distribution-free, so you can't just simulate the distribution and be done. If you want tables for specific distributional assumptions (under some location-family), you could do those.

On the other hand, if you have a parametric model, you could manage this using more standard methods.

You might perhaps do something with permutation / boostrapping of residuals from a robust estimate of location-difference, but it would be approximate; it might often be adequate in largish samples though. I haven't tried this, though so I can't say a lot about its likely performance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.