2
$\begingroup$

I am comparing three distributions of values. Let's say A, B and C.

If I find that the p-value of the Wilcoxon rank sum test between A and B is 1e-08 and between A and C is 1e-10, can I say that distribution A is more similar to B than it is to C?

$\endgroup$
9
  • $\begingroup$ Given those values, I don't think there is much meaning left in saying A is more similar to either one. $\endgroup$ Sep 18 '13 at 16:44
  • $\begingroup$ 'Wilcoxon rank-sum test'? $\endgroup$ Sep 18 '13 at 16:44
  • 1
    $\begingroup$ Scortchi, yes it's a Wilcoxon rank-sum test. $\endgroup$
    – panos
    Sep 19 '13 at 6:59
  • 1
    $\begingroup$ Glen_b, I'm using it in order to compare if distributions of values differ from each other or not. More specifically, I'm studying the length of introns between different organisms and want to see if the introns from one organism are significantly longer/shorter compared to another. $\endgroup$
    – panos
    Sep 19 '13 at 7:05
  • 1
    $\begingroup$ I don't feel completely comfortable with the use of the term "distribution" in the question. Most inferential tests (like Wilcoxon) compares the central tendency, not the shape of the distribution. We're dealing with ordinal type data, so it is probably not as important to consider this. Still, for example, B and C could have the exact same median but B could be bimodal and C could be unimodal. Something to think about? $\endgroup$
    – Hotaka
    Sep 19 '13 at 15:26
7
$\begingroup$

No. According to Gelman and Stern (2012), "even large changes in significance levels can correspond to small, nonsignificant changes in the underlying quantities." In plain English, differences between p-values are generally not statistically significant.

$\endgroup$
1
$\begingroup$

No. To say the least, it would depend on sample size. More importantly, why would you want to do that? I'd simply do 3 tests to compare all 3 with each other using Bonferroni correction or similar.

But if you're dealing with length, why not ANOVA??

$\endgroup$
2
  • 1
    $\begingroup$ I believe it's not possible to get p-values that low on the Wilcoxon test unless the sample sizes are pretty large (or at least of moderate size--$10$ or greater--and the differences are as extreme as possible). But given that the differences are stated in terms of p-values and not effect sizes, why and how would any such conclusion depend on sample size? $\endgroup$
    – whuber
    Sep 19 '13 at 7:29
  • $\begingroup$ Well, the test is a between-subjects comparison, so A, B, and C could have differing sample sizes. Looking at the p-values, it may seem like A and C are further apart than A and B, but it might just mean they are both the same distance and C just has a higher sample size. $\endgroup$
    – Hotaka
    Sep 19 '13 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.