Return to Revisions

With many measurements (like 995), small differences may already become significant. A small effect size can still be significant.

Also a t-test will show a significant difference. The means have a difference of about $0.3$ and the t-statistic is around $2.4$ giving a p-value of around $0.0166$

Are smaller p-values more convincing?

Why is "statistically significant" not enough?

Btw, the Mann-Whitney U test is not the same as a t-test and can be significant, even when the means are the same (https://stats.stackexchange.com/a/470512/).

The MWU test is testing whether $P(X>Y) = P(Y>X)$. This is the case when in a PP-plot the curve divides the plane into two equal areas. So, in relation to a MWU test, a PP-plot might be a better way to visualize the difference rather than two histograms.