# Compute significance of Kendall tau-b?

Compute significance of Kendall tau-b?

I have so-far tried all ways of computing kendall tau significance (where there are ties) described here.

However, none of them works good, even for relatively large vectors. I think the problem is in general it seems that these methods tend to overestimate the statistical significance of data, so are there any alternative ways to compute kendall tau's significance? (at least get some degree of tie-adjustments), many thanks.

-
Could you tell us how exactly the adjustments don't work? –  Stephan Kolassa Jan 9 '13 at 8:29
@Stephan Kolassa I compare the significance computed by these methods with the exact significance level computed from permutations. –  user55647 Jan 9 '13 at 8:32
In that case, could you just use that permutation test to assess significance? For larger vectors, you could probably not go through all permutations, but 10,000 random permutations are often already enough. –  Stephan Kolassa Jan 9 '13 at 8:36
Sure, but the computational cost is huge (in some cases when you may not satisified with some 0.05 significance level). –  user55647 Jan 9 '13 at 8:46
@StephanKolassa Maybe there's something I am missing, but it seems fairly straightforward to generate the most extreme ones. That's what the various specialized 'exact' permutation test algorithms try to do (eg various papers by Mehta, Patel and some of their coauthors, as well as a number of others) - specifically to move through the extreme permutations of various statistics in order from most extreme, in efficient ways. Is there a reason why such approaches can't be used/adapted to work with tau-b? Ties complicate it, but I don't think they'll prevent those approaches from being used. –  Glen_b Jan 10 '13 at 1:10

Note that since the denominator only depends on the margins, not on the association, the exact permutation p-value is the same for any of the statistics that has $N_c-N_d$ on the numerator (Kendall's tau, tau-b, tau-c, Somer's-D etc).
(Out of curiosity, why would it matter for you if you were only able to get a probabilistic bound on the p-value using resampling? How does (say) knowing the p-value is almost certainly less that $3.2 \times 10^{-5}$ rather than computing it to be exactly $1.6245 \times 10^{-6}$? What additional information does that give you?)