Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Compute significance of Kendall tau-b?

I have so-far tried all ways of computing kendall tau significance (where there are ties) described here.

However, none of them works good, even for relatively large vectors. I think the problem is in general it seems that these methods tend to overestimate the statistical significance of data, so are there any alternative ways to compute kendall tau's significance? (at least get some degree of tie-adjustments), many thanks.

share|improve this question
    
Could you tell us how exactly the adjustments don't work? –  Stephan Kolassa Jan 9 '13 at 8:29
    
@Stephan Kolassa I compare the significance computed by these methods with the exact significance level computed from permutations. –  user55647 Jan 9 '13 at 8:32
    
In that case, could you just use that permutation test to assess significance? For larger vectors, you could probably not go through all permutations, but 10,000 random permutations are often already enough. –  Stephan Kolassa Jan 9 '13 at 8:36
    
Sure, but the computational cost is huge (in some cases when you may not satisified with some 0.05 significance level). –  user55647 Jan 9 '13 at 8:46
1  
@StephanKolassa Maybe there's something I am missing, but it seems fairly straightforward to generate the most extreme ones. That's what the various specialized 'exact' permutation test algorithms try to do (eg various papers by Mehta, Patel and some of their coauthors, as well as a number of others) - specifically to move through the extreme permutations of various statistics in order from most extreme, in efficient ways. Is there a reason why such approaches can't be used/adapted to work with tau-b? Ties complicate it, but I don't think they'll prevent those approaches from being used. –  Glen_b Jan 10 '13 at 1:10
show 9 more comments

1 Answer 1

up vote 1 down vote accepted

Note that since the denominator only depends on the margins, not on the association, the exact permutation p-value is the same for any of the statistics that has $N_c-N_d$ on the numerator (Kendall's tau, tau-b, tau-c, Somer's-D etc).

There are algorithms for computing p-values of all these measures of association for ordinal data efficiently, by taking account of the ordering of the permutations induced by some simple form of the test statistic, and only considering the ones more extreme than the test statistic (or considering as few additional ones as possible), generally arising from ideas based on Mehta and Patel's networking algorithm, though there have been developments and ideas from many authors.

Some stats packages implement such algorithms. It might be worth checking whether some of the exact-test related packages already have an efficient version of it for some version of the Kendall statistic.

For example, I believe SPSS Exact Tests has this implemented for the Kendall tau.

(Out of curiosity, why would it matter for you if you were only able to get a probabilistic bound on the p-value using resampling? How does (say) knowing the p-value is almost certainly less that $3.2 \times 10^{-5}$ rather than computing it to be exactly $1.6245 \times 10^{-6}$? What additional information does that give you?)

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.