I want to determine if there's a difference in mean p-values between two groups. In order to do this I perform a Wilcoxon's rank-sum test (the data is not normally distributed). So far, so good. Finally, I want to calculate the corresponding effect size. Unfortunately, R does not provide this. It also does not provide a z value with which the effect size can easily be calculated using: effect size = z / sqrt(N)

here is some sample R code:

a=rep(0:1,each=20)                            #grouping variable
b=c(rnorm(20, .03,.01), rnorm(20, .02, .009)) #vector of p-values


test = wilcox.test(b ~ a, data = d)     #perform Wilcoxon rank-sum test

Does anybody know how to obtain the effect size?

  • 3
    $\begingroup$ Welcome to SO. I flagged your question for migration to www.crossvalidated.com , as your question is more statistical than anything else. In short : Wilcoxon RANK test works with ranks, so I'm not sure about which effect size you're talking. Obviously it doesn't give you a z value, as that one is linked to parametric testing, not to non-parametric tests like Wilcoxon. Wilcoxon has to be interpreted in terms of location shift. $\endgroup$
    – Joris Meys
    Jun 14, 2012 at 14:47
  • 6
    $\begingroup$ The Wilcoxon rank sum test does not evaluate a difference in means: it tests for stochastic inequality. The proper measure of an effect size therefore is the probability that $B \gt A$, minus the null value of $1/2$. $\endgroup$
    – whuber
    Jun 14, 2012 at 15:07
  • $\begingroup$ @whuber And how is this effect size calculated? From what I've read, it must be a number between 0 and 1 (or between -1 and 1?), but the most voted answer provides a different number (-6 in my case, even divided by sqrt(N) it's still < -1). $\endgroup$
    – Rodrigo
    Nov 13, 2020 at 10:30
  • $\begingroup$ Rodrigo, please see stats.stackexchange.com/questions/133077/… and stats.stackexchange.com/questions/342987/…. $\endgroup$
    – whuber
    Nov 13, 2020 at 14:23

2 Answers 2


The estimator that corresponds to the Wilcoxon test is the Hodges-Lehmann estimator; it's returned by wilcox.test using the conf.int=TRUE option, under "difference in location".

For your example:

> wilcox.test(b~a,data=d, conf.int=TRUE)

    Wilcoxon rank sum test

data:  b by a 
W = 355, p-value = 6.914e-06
alternative hypothesis: true location shift is not equal to 0 
95 percent confidence interval:
 0.008657301 0.021523993 
sample estimates:
difference in location 

For more on the Wilcoxon and the assumptions behind it, and what it actually tests for, and other nonparametric estimators, this document is (possibly) helpful: www.stat.umn.edu/geyer/old03/5102/notes/rank.pdf

  • $\begingroup$ So what you say is that the effect size is wilcox.test(b~a,data=d, conf.int=TRUE)$estimate / sqrt(20), correct? $\endgroup$
    – user14650
    Jan 9, 2015 at 13:29

Get the z for your formula by

mydf <- as.data.frame(d)
wilcoxsign_test(b ~ a, data = mydf, distribution="exact")

and compute the effect size with your formula, setting N to 40

  • 1
    $\begingroup$ I suspect this reply would be of higher interest if you could supply additional details on the reasons to use the coin::wilcoxsign_test R function. Also, are you referring to the OP formula, $\text{effect size} = z / \sqrt{N}$? $\endgroup$
    – chl
    Apr 4, 2013 at 19:52
  • $\begingroup$ The reason ti use coin::wilcoxsign_test is that it computes a z value. $\endgroup$ Apr 5, 2013 at 9:43
  • 1
    $\begingroup$ Another way to obtain z from standard wilcox.test is the algorithm Andy Fields implemented in his function rFromWilcox. And yes, I was referring to effect size = z / sqrt(N). $\endgroup$ Apr 5, 2013 at 18:17
  • 2
    $\begingroup$ Thanks for the clarification although I'm still unsure about that z-value being a proper measure of effect size. (+1) $\endgroup$
    – chl
    Apr 5, 2013 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.