4
$\begingroup$

I have so-far tried all ways of computing kendall tau significance (where there are ties) described.

However, none of them works good, even for relatively large vectors. I think the problem is in general it seems that these methods tend to overestimate the statistical significance of data, so are there any alternative ways to compute kendall tau's significance? (at least get some degree of tie-adjustments)?

Improve this question
$\endgroup$
14
  • $\begingroup$ Could you tell us how exactly the adjustments don't work? $\endgroup$
    – Stephan Kolassa
    Commented Jan 9, 2013 at 8:29
  • $\begingroup$ @Stephan Kolassa I compare the significance computed by these methods with the exact significance level computed from permutations. $\endgroup$
    – user55647
    Commented Jan 9, 2013 at 8:32
  • $\begingroup$ In that case, could you just use that permutation test to assess significance? For larger vectors, you could probably not go through all permutations, but 10,000 random permutations are often already enough. $\endgroup$
    – Stephan Kolassa
    Commented Jan 9, 2013 at 8:36
  • $\begingroup$ Sure, but the computational cost is huge (in some cases when you may not satisified with some 0.05 significance level). $\endgroup$
    – user55647
    Commented Jan 9, 2013 at 8:46
  • 1
    $\begingroup$ @StephanKolassa Maybe there's something I am missing, but it seems fairly straightforward to generate the most extreme ones. That's what the various specialized 'exact' permutation test algorithms try to do (eg various papers by Mehta, Patel and some of their coauthors, as well as a number of others) - specifically to move through the extreme permutations of various statistics in order from most extreme, in efficient ways. Is there a reason why such approaches can't be used/adapted to work with tau-b? Ties complicate it, but I don't think they'll prevent those approaches from being used. $\endgroup$
    – Glen_b
    Commented Jan 10, 2013 at 1:10

1 Answer 1

Reset to default
2
$\begingroup$

Note that since the denominator only depends on the margins, not on the association, the exact permutation p-value is the same for any of the statistics that has $N_c-N_d$ on the numerator (Kendall's tau, tau-b, tau-c, Somer's-D etc).

There are algorithms for computing p-values of all these measures of association for ordinal data efficiently, by taking account of the ordering of the permutations induced by some simple form of the test statistic, and only considering the ones more extreme than the test statistic (or considering as few additional ones as possible), generally arising from ideas based on Mehta and Patel's networking algorithm, though there have been developments and ideas from many authors.

Some stats packages implement such algorithms. It might be worth checking whether some of the exact-test related packages already have an efficient version of it for some version of the Kendall statistic.

For example, I believe SPSS Exact Tests has this implemented for the Kendall tau.

(Out of curiosity, why would it matter for you if you were only able to get a probabilistic bound on the p-value using resampling? How does (say) knowing the p-value is almost certainly less that $3.2 \times 10^{-5}$ rather than computing it to be exactly $1.6245 \times 10^{-6}$? What additional information does that give you?)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.