I have a paired data with a small sample size of 19 subjects who gave different scores at time 1 ($T_1$) and time 2 ($T_2$). I'd like to conduct a Wilcoxon test using R, because my data are not normal.
wilcox.test gives a warning message that says that exact p-value cannot be computed with ties and zeroes.
I have only 4 differences ($T_1-T_2$) out of 19 that are equal to 0. Can I still use this test?
The purpose of this answer is to illustrate some possible tests for your data. Suppose data are as follows:
x1 = c(5, 6, 3, 3, 4, 2, 5, 3, 4, 3, 4, 5, 3, 2, 2, 5, 4, 1, 3)
x2 = c(4, 5, 3, 2, 2, 1, 4, 5, 4, 2, 2, 4, 2, 1, 3, 5, 3, 1, 2)
Wilcoxon signed rank test. A paired Wilcoxon test is a one-sample Wilcoxon signed rank on differences. Differences for the data above are:
d = x1-x2; d
[1] 1 1 0 1 2 1 1 -2 0 1 2 1 1 1 -1 0 1 0 1
hdr = "Stripchart of Differences"
stripchart(d, meth="stack", pch=20, ylim=c(.5, 2), main=hdr)
So there are four 0s and many ties.
wilcox.test(d)
Wilcoxon signed rank test with continuity correction
data: d
V = 99.5, p-value = 0.01842
alternative hypothesis: true location is not equal to 0
Warning messages:
1: In wilcox.test.default(d) : cannot compute exact p-value with ties
2: In wilcox.test.default(d) : cannot compute exact p-value with zeroes
The questionable P-value about 0.02 is below 5%. Just to get an idea how seriously wrong the P-value might be (not as a formal test), you could use slight jittering to avoid ties and zeros.
j = runif(19, -.01, .01)
dj = d+j; wilcox.test(dj)$p.val
[1] 0.004577637
j = runif(19, -.01, .01)
dj = d+j; wilcox.test(dj)$p.val
[1] 0.04455948
j = runif(19, -.01, .01)
dj = d+j; wilcox.test(dj)$p.val
[1] 0.02582169
For my taste, this is enough of a clue that there may be a significant difference below the 5% level. It seems worthwhile to look at alternative tests.
Sign test for median of differences. One alternative test for the null hypothesis that the mean is $0$ is a sign test. The 0s contribute no useful information; out of the 15 non-zero
differences, 13 are positive and 2 are negative. So the P-value of a one-sided sign test is judged inconsistent with $0$ median, with P-value about 0.004.
pbinom(2, 15, .5)
[1] 0.003692627
Sign tests ignore how much positive or negative the differences are, so their power is often not very good.
Permutation test for mean differences. Another alternative would be to do a permutation test, using the mean difference as metric. We randomly permute the signs of the 19 observations many times, find the mean each time, and compare the observed mean difference with the simulated permutation distribution of means. [The mean difference is an appropriate metric if we believe the original data are numerical (rather than labels of ordinal categories).]
The P-value (about 0.012) of a one-sided test is significant.
set.seed(319)
a.prm = replicate(10^4, mean(sample(c(-1,1), 19, rep=T)*d))
mean(a.prm >= a.obs)
[1] 0.0118 # P-val for one-sided test
mean(abs(a.prm) > abs(a.obs))
[[1] 0.0231 # P-val for two-sided test
length(unique(a.prm))
[1] 17
There are 17 uniquely different permuted mean differences, as shown in the histogram below, along with the observed mean difference (vertical red line).
The Wilcoxon is performing a ranking and when there is a tie, there is not a method to properly assign one over the other.
For example, assume your data is x=(1, 2) and y=(2, 3) when you rank these values it could be (x1, x2, y2, y3) or (x1, y2, x2, y3). Both rankings are equally valid but the the calculated statistic is different. Because of this arbitrary ranking the warning is provided. If you dataset is large enough, with a few ties, it should all balance out.
For your case with your case, it might be more borderline depending on the p-value. You could try rerunning the test a few times switching the ordering of T1 & T2 to see if the p-value changes. Also, try running the test with the ties removed.
Good Luck.
