I'm using the Mann-Whitney U test to compare scores from a foreground (n ranging between 5-500) to a large background (up to 2000). Since my foreground is quite small I've been taking smaller random samples of my background and comparing the same foreground to multiple background sets. I'd like to have a way of summarizing these tests into 1 p-value but I'm at a loss of anyway to do this besides Fisher's method. I know this isn't correct because the tests are not independent of each other as they have the same background. Is there any way to summarize these p-values correctly that allows for this sort of analysis?

  • $\begingroup$ It sounds like you are throwing away data in order to be able to run a given statistical test on the data. In general, throwing away data is wasteful and not a good idea. Can you just scale the background scores in some logical fashion? $\endgroup$
    – Joel W.
    Jan 14, 2015 at 18:32
  • 1
    $\begingroup$ The answer is yes: run the U test between all foreground and all background data. $\endgroup$
    – whuber
    Aug 5, 2018 at 14:07


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.