Why don't my confidence interval and p-value match for Kendall's Tau?

Question

I'm trying to examine the relationship between two samples of ordinal scale values, by computing Kendall's Tau and its corresponding confidence interval (CI) and p-value.

I used the R function cor.test (base package) to calculate the p-value:

> cor.test(x, y, alternative = "two.sided", method = "kendall")

which returns:

data:  x and y
z = -1.8504, p-value = 0.06425
alternative hypothesis: true tau is not equal to 0
sample estimates:
        tau 
-0.02553355 

and I used the R function kendall.ci (NSM3 package) to calculate the CI:

> kendall.ci(x, y, alpha=0.05, type="t")

which returns:

1 - alpha = 0.95 two-sided CI for tau:
-0.042, -0.009

The problem: I'm having trouble interpreting and reporting these results, as I'd expect a 95% CI that does not include zero (the null-hypothesis value) to correspond to a p-value lower than 0.05, or conversely, a p-value above 0.05 with a 95% CI that includes zero.

Re the data: x and y both have 4,081 integer elements, no NaN's and 6 and 10 unique values, respectively (i.e. many "recurring values" or "ties"). I suspect the issue may be related to how these functions handle ties, but have yet to find the answer in the docs.

Any help would be appreciated.

Thanks!

Answer 1

Looking at the code for kendall.ci, it seems to be using a U-statistic formula for the variance, which will be correct only for continuous distributions (though it should be an ok approximation more generally).Since you have a lot of ties and the $p$-value only just fails to match the confidence interval, I think that's the issue.

According to the documentation, the kendall.ci function has a bootstrap option, which should give you better results.