8
$\begingroup$

I want to determine if there's a difference in mean p-values between two groups. In order to do this I perform a Wilcoxon's rank-sum test (the data is not normally distributed). So far, so good. Finally, I want to calculate the corresponding effect size. Unfortunately, R does not provide this. It also does not provide a z value with which the effect size can easily be calculated using: effect size = z / sqrt(N)

here is some sample R code:

a=rep(0:1,each=20)                            #grouping variable
b=c(rnorm(20, .03,.01), rnorm(20, .02, .009)) #vector of p-values

d=cbind(a,b)

test = wilcox.test(b ~ a, data = d)     #perform Wilcoxon rank-sum test
test

Does anybody know how to obtain the effect size?

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ Welcome to SO. I flagged your question for migration to www.crossvalidated.com , as your question is more statistical than anything else. In short : Wilcoxon RANK test works with ranks, so I'm not sure about which effect size you're talking. Obviously it doesn't give you a z value, as that one is linked to parametric testing, not to non-parametric tests like Wilcoxon. Wilcoxon has to be interpreted in terms of location shift. $\endgroup$
    – Joris Meys
    Commented Jun 14, 2012 at 14:47
  • 7
    $\begingroup$ The Wilcoxon rank sum test does not evaluate a difference in means: it tests for stochastic inequality. The proper measure of an effect size therefore is the probability that $B \gt A$, minus the null value of $1/2$. $\endgroup$
    – whuber ♦
    Commented Jun 14, 2012 at 15:07
  • $\begingroup$ @whuber And how is this effect size calculated? From what I've read, it must be a number between 0 and 1 (or between -1 and 1?), but the most voted answer provides a different number (-6 in my case, even divided by sqrt(N) it's still < -1). $\endgroup$
    – Rodrigo
    Commented Nov 13, 2020 at 10:30
  • $\begingroup$ Rodrigo, please see stats.stackexchange.com/questions/133077/… and stats.stackexchange.com/questions/342987/…. $\endgroup$
    – whuber ♦
    Commented Nov 13, 2020 at 14:23

2 Answers 2

Reset to default
13
$\begingroup$

The estimator that corresponds to the Wilcoxon test is the Hodges-Lehmann estimator; it's returned by wilcox.test using the conf.int=TRUE option, under "difference in location".

For your example:

> wilcox.test(b~a,data=d, conf.int=TRUE)

    Wilcoxon rank sum test

data:  b by a 
W = 355, p-value = 6.914e-06
alternative hypothesis: true location shift is not equal to 0 
95 percent confidence interval:
 0.008657301 0.021523993 
sample estimates:
difference in location 
            0.01442617

For more on the Wilcoxon and the assumptions behind it, and what it actually tests for, and other nonparametric estimators, this document is (possibly) helpful: www.stat.umn.edu/geyer/old03/5102/notes/rank.pdf

Improve this answer
$\endgroup$
2
  • $\begingroup$ So what you say is that the effect size is wilcox.test(b~a,data=d, conf.int=TRUE)$estimate / sqrt(20), correct? $\endgroup$
    – user14650
    Commented Jan 9, 2015 at 13:29
  • $\begingroup$ One of the implications of the excellent Geyer notes is that the pseudomedian (Hodges-Lehman one-sample estimator) is not suitable for asymmetric distributions. I don't think that is quite correct. $\endgroup$
    – Frank Harrell
    Commented Oct 30 at 16:03
2
$\begingroup$

Get the z for your formula by

library(coin)
mydf <- as.data.frame(d)
wilcoxsign_test(b ~ a, data = mydf, distribution="exact")

and compute the effect size with your formula, setting N to 40

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ I suspect this reply would be of higher interest if you could supply additional details on the reasons to use the coin::wilcoxsign_test R function. Also, are you referring to the OP formula, $\text{effect size} = z / \sqrt{N}$? $\endgroup$
    – chl
    Commented Apr 4, 2013 at 19:52
  • $\begingroup$ The reason ti use coin::wilcoxsign_test is that it computes a z value. $\endgroup$
    – Primigenius
    Commented Apr 5, 2013 at 9:43
  • 1
    $\begingroup$ Another way to obtain z from standard wilcox.test is the algorithm Andy Fields implemented in his function rFromWilcox. And yes, I was referring to effect size = z / sqrt(N). $\endgroup$
    – Primigenius
    Commented Apr 5, 2013 at 18:17
  • 2
    $\begingroup$ Thanks for the clarification although I'm still unsure about that z-value being a proper measure of effect size. (+1) $\endgroup$
    – chl
    Commented Apr 5, 2013 at 20:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.