2
$\begingroup$

I'm using wilcox.test to compare two timepoints as such:

a <- c(1,2,3,4,5)
b <- c(2,3,4,5,6)
wilcox.test(a, b)

The result of the rank sum test is, as expected p > 0.05 (0.3976) If I treat the samples as paired, the result is, as expected more significant:

wilcox.test(a, b, paired=T)

with the signed rank test coming up with a p-value of 0.007937.

But I don't understand why the same doesn't hold true with actual data from my experiment:

a <- c(244.5, 242.3, 225.25, 250.15, 254.0)
b <- c(196.8, 186.25, 175.75, 174.75, 170.33)
wilcox.test(a, b)

Rank sum test: W = 25, p-value = 0.007937

wilcox.test(a,b, paired=T)

Signed rank test: V = 15, p-value = 0.0625

I don't quite understand why the latter is less powerful in this example. Is there a better test to use to test the hypothesis that the measurements differ between these two timepoints (without assuming a normal distribution)?

$\endgroup$
  • 1
    $\begingroup$ Your example data have ties. In the case of the rank sum test (unpaired), all observations are paired together, so the values 2 through 5 are tied, and as wilcox.test reports, it cannot compute exact p-value with ties. The same is true of the sign rank test, where, for your example data, the ranked differences are all equal (to 1, as it happens). Again, R reports cannot compute exact p-value with ties. One would expect the sign rank test (paired) to produce lower p-values, as in your actual data, all other things being equal. $\endgroup$ – Alexis Mar 24 '15 at 2:39
  • $\begingroup$ Right, sorry about the ties in the dummy example, but the actual data example doesn't have any ties. And the test on the actual data produces a worse (greater) p-value in the paired case. It can't be related to ties. $\endgroup$ – reviewer3 Mar 24 '15 at 2:44
  • 1
    $\begingroup$ Dang it... I misread your actual data output first time around. Ok... second try: the paired test has $n=5$, while the unpaired test has $n=10$. With your actual data, the differences in rank sums (unpaired), and rank of differences (paired) is at the maximum possible extremity (group a dominates over group b, always). With $n=5$, the smallest possible $p$-value in a paired test is 0.0625. With $n=10$ ($n=5$ in both groups) smallest possible p-value for the unpaired test is quite a bit smaller. $\endgroup$ – Alexis Mar 24 '15 at 3:08
  • $\begingroup$ Alexis, that makes sense to me! I knew it had to be related to the difference in the tests (signed rank vs. rank sum) but didn't consider the n's differ. I upvoted your comment - thank you. Any idea on a good alternative to do a paired analysis with only 5 pairs? Thank you! $\endgroup$ – reviewer3 Mar 24 '15 at 14:21
3
$\begingroup$

Paired tests leverage the correlation between your variables (typically before and after) to improve power. The more strongly, and positively, your data are correlated, the more of the variability that exists between the units (e.g., patients), and which gets ignored by the test (although if that variability would have otherwise supported the alternative hypothesis, this looses power, see comments below). If the data were uncorrelated, there shouldn't be any long run gain, and if the data were negatively correlated, taking that fact into account (which is the correct thing to do) shows that there is actually less information in your data than might naively appear. Let's look at your data:

a <- c(1,2,3,4,5)
b <- c(2,3,4,5,6)
cor(a, b)
# [1] 1
wilcox.test(a,b)
#  Wilcoxon rank sum test with continuity correction
# 
# data:  a and b
# W = 8, p-value = 0.3976
# alternative hypothesis: true location shift is not equal to 0
wilcox.test(a,b, paired=T)
#  Wilcoxon signed rank test with continuity correction
# 
# data:  a and b
# V = 0, p-value = 0.03689
# alternative hypothesis: true location shift is not equal to 0


a <- c(244.5, 242.3, 225.25, 250.15, 254.0)
b <- c(196.8, 186.25, 175.75, 174.75, 170.33)
cor(a, b)
# [1] -0.1026423
wilcox.test(a, b)
#  Wilcoxon rank sum test
# 
# data:  a and b
# W = 25, p-value = 0.007937
# alternative hypothesis: true location shift is not equal to 0
wilcox.test(a, b, paired=T)
#  Wilcoxon signed rank test
# 
# data:  a and b
# V = 15, p-value = 0.0625
# alternative hypothesis: true location shift is not equal to 0
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you, but there's still something I don't understand. Consider this better dummy example: a<-seq(1,5); b<-a+runif(5,0,1); wilcox.test(a,b, paired=T) The resulting p-value is once again 0.0625, even though the PCC is close 1 in this case. $\endgroup$ – reviewer3 Mar 24 '15 at 2:55
  • $\begingroup$ I don't follow your new example. seq(1,5) is the same as c(1,2,3,4,5). $\endgroup$ – gung - Reinstate Monica Mar 24 '15 at 2:56
  • $\begingroup$ And when I increase the difference between a & b the unpaired test wins out, while the paired test always reproduces a p-value of 0.0625. a<-seq(1,5); b<-a+runif(5,0,1)+10; wilcox.test(a,b, paired=F) $\endgroup$ – reviewer3 Mar 24 '15 at 3:00
  • $\begingroup$ @reviewer3, you need to set the seed for reproducibility when you call a (pseudo) random function. I did set.seed(8695), then ran your top code. The r = 0.9959916, unpaired M-W U-test: p-value = 0.6905, whereas paired Wilcoxon is p-value = 0.0625. This is consistent w/ the strong positive r. $\endgroup$ – gung - Reinstate Monica Mar 24 '15 at 3:03
  • 1
    $\begingroup$ Looking at correlation is good, even better with a rank correlation ... > cor(a,b,method="spearman") ... [1] -0.6 There's the problem. $\endgroup$ – Glen_b Mar 24 '15 at 3:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.