Compute significance of Kendall tau-b?

I have so-far tried all ways of computing kendall tau significance (where there are ties) described here.

However, none of them works good, even for relatively large vectors. I think the problem is in general it seems that these methods tend to overestimate the statistical significance of data, so are there any alternative ways to compute kendall tau's significance? (at least get some degree of tie-adjustments), many thanks.

  • $\begingroup$ Could you tell us how exactly the adjustments don't work? $\endgroup$ – S. Kolassa - Reinstate Monica Jan 9 '13 at 8:29
  • $\begingroup$ @Stephan Kolassa I compare the significance computed by these methods with the exact significance level computed from permutations. $\endgroup$ – user55647 Jan 9 '13 at 8:32
  • $\begingroup$ In that case, could you just use that permutation test to assess significance? For larger vectors, you could probably not go through all permutations, but 10,000 random permutations are often already enough. $\endgroup$ – S. Kolassa - Reinstate Monica Jan 9 '13 at 8:36
  • $\begingroup$ Sure, but the computational cost is huge (in some cases when you may not satisified with some 0.05 significance level). $\endgroup$ – user55647 Jan 9 '13 at 8:46
  • 1
    $\begingroup$ @StephanKolassa Maybe there's something I am missing, but it seems fairly straightforward to generate the most extreme ones. That's what the various specialized 'exact' permutation test algorithms try to do (eg various papers by Mehta, Patel and some of their coauthors, as well as a number of others) - specifically to move through the extreme permutations of various statistics in order from most extreme, in efficient ways. Is there a reason why such approaches can't be used/adapted to work with tau-b? Ties complicate it, but I don't think they'll prevent those approaches from being used. $\endgroup$ – Glen_b -Reinstate Monica Jan 10 '13 at 1:10

Note that since the denominator only depends on the margins, not on the association, the exact permutation p-value is the same for any of the statistics that has $N_c-N_d$ on the numerator (Kendall's tau, tau-b, tau-c, Somer's-D etc).

There are algorithms for computing p-values of all these measures of association for ordinal data efficiently, by taking account of the ordering of the permutations induced by some simple form of the test statistic, and only considering the ones more extreme than the test statistic (or considering as few additional ones as possible), generally arising from ideas based on Mehta and Patel's networking algorithm, though there have been developments and ideas from many authors.

Some stats packages implement such algorithms. It might be worth checking whether some of the exact-test related packages already have an efficient version of it for some version of the Kendall statistic.

For example, I believe SPSS Exact Tests has this implemented for the Kendall tau.

(Out of curiosity, why would it matter for you if you were only able to get a probabilistic bound on the p-value using resampling? How does (say) knowing the p-value is almost certainly less that $3.2 \times 10^{-5}$ rather than computing it to be exactly $1.6245 \times 10^{-6}$? What additional information does that give you?)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.