I have two samples both bigger than 99999 and I get the following result in Python:

MannwhitneyuResult(statistic=2395063024.0, pvalue=2.3093898533164459e-128)

U critical = 135807.32

If I understand it correctly, I cannot reject the null hypothesis that samples (distributions) are the same because U obtained > U critical.

However, If I make boxplots it seems like one distribution is bigger than the other one.

enter image description here

enter image description here

Are my results correct and what can I conclude?

  • $\begingroup$ Something is wrong. The p-value and critical value do not make sense. Besides the a p-value that low would mean that the observed test statistic is way beyond the critical value when a significance level like 0.05 or 0.01 is used. $\endgroup$ – Michael R. Chernick Mar 23 '18 at 1:02
  • $\begingroup$ Ah, I see that on the page for scipy.stats.mannwhitneyu (I presume that's what you're calling then) that it says that their version of the statistic is significant if it's smaller than the critical value. However, there's a comment higher up that suggests that it may depend on the alternative chosen. You don't say what alternative you actually used. $\endgroup$ – Glen_b Mar 23 '18 at 1:34
  • $\begingroup$ Thank you both. I am using scipy.stats.mannwhitneyu. I haven't used the alternative since it was optional but I will try it out. $\endgroup$ – Ami Mar 23 '18 at 12:05

Unfortunately, the conclusion is reversed.  Because $p \ll \alpha$, you would reject the null (which assumes the distributions have the same parameter).  Additionally, the sample size all but guarantees that you will reject the null.  In this case, it may be more beneficial to accept that the distributions are different and address the difference between the two samples from a practical significance (an effect size) instead of a statistical significance.

  • $\begingroup$ Thank you very much. I will do that. I just have one more question. Would it be ok to conclude which one is bigger based just on the boxplots? $\endgroup$ – Ami Mar 23 '18 at 12:01
  • $\begingroup$ You could conclude that by comparing the boxplots, or the medians, or other target percentiles of interest (e.g., 75th, 90th, 99th, etc.) $\endgroup$ – Gregg H Mar 23 '18 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.