2
$\begingroup$

I am comparing three distributions of values. Let's say A, B and C.

If I find that the p-value of the Wilcoxon rank sum test between A and B is 1e-08 and between A and C is 1e-10, can I say that distribution A is more similar to B than it is to C?

Improve this question
$\endgroup$
9
  • $\begingroup$ Given those values, I don't think there is much meaning left in saying A is more similar to either one. $\endgroup$
    – Marc Claesen
    Commented Sep 18, 2013 at 16:44
  • $\begingroup$ 'Wilcoxon rank-sum test'? $\endgroup$
    – Scortchi ♦
    Commented Sep 18, 2013 at 16:44
  • 1
    $\begingroup$ Scortchi, yes it's a Wilcoxon rank-sum test. $\endgroup$
    – panos
    Commented Sep 19, 2013 at 6:59
  • 1
    $\begingroup$ Glen_b, I'm using it in order to compare if distributions of values differ from each other or not. More specifically, I'm studying the length of introns between different organisms and want to see if the introns from one organism are significantly longer/shorter compared to another. $\endgroup$
    – panos
    Commented Sep 19, 2013 at 7:05
  • 1
    $\begingroup$ I don't feel completely comfortable with the use of the term "distribution" in the question. Most inferential tests (like Wilcoxon) compares the central tendency, not the shape of the distribution. We're dealing with ordinal type data, so it is probably not as important to consider this. Still, for example, B and C could have the exact same median but B could be bimodal and C could be unimodal. Something to think about? $\endgroup$
    – Hotaka
    Commented Sep 19, 2013 at 15:26

2 Answers 2

Reset to default
7
$\begingroup$

No. According to Gelman and Stern (2012), "even large changes in significance levels can correspond to small, nonsignificant changes in the underlying quantities." In plain English, differences between p-values are generally not statistically significant.

Improve this answer
$\endgroup$
1
$\begingroup$

No. To say the least, it would depend on sample size. More importantly, why would you want to do that? I'd simply do 3 tests to compare all 3 with each other using Bonferroni correction or similar.

But if you're dealing with length, why not ANOVA??

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ I believe it's not possible to get p-values that low on the Wilcoxon test unless the sample sizes are pretty large (or at least of moderate size--$10$ or greater--and the differences are as extreme as possible). But given that the differences are stated in terms of p-values and not effect sizes, why and how would any such conclusion depend on sample size? $\endgroup$
    – whuber ♦
    Commented Sep 19, 2013 at 7:29
  • $\begingroup$ Well, the test is a between-subjects comparison, so A, B, and C could have differing sample sizes. Looking at the p-values, it may seem like A and C are further apart than A and B, but it might just mean they are both the same distance and C just has a higher sample size. $\endgroup$
    – Hotaka
    Commented Sep 19, 2013 at 15:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.