Does it "rarely make sense" to compute Kendall's $\tau$ for a large sample?

Question

The manual page for R's cor says:

Some people have noted that the code for Kendall's tau is slow for very large datasets (many more than 1000 cases). It rarely makes sense to do such a computation, but see function cor.fk in package pcaPP.

Why wouldn't it make sense to compute a Kendall's $\tau$ for a large sample? Is there some reason $\tau$ is less useful or meaningful with larger samples? Or is it just that $\tau$ is hard to compute and you might as well approximate it by randomly sampling pairs of points and checking how often they agree?

Answer 1

Here's my take on things:

Why wouldn't it make sense to compute a Kendall's τ for a large sample?

If time is really an issue, there are other options for nonparametric correlation statistics. From my understanding, Kendall's tau was meant for small samples (<100 observations). So why not use Spearman's rank?

SPSS documentation has some guidance for this: https://statistics.laerd.com/spss-tutorials/kendalls-tau-b-using-spss-statistics.php

In general, you may not want to compute Kendall's tau if time is a serious issue. There are other alternatives that have a lower computation time such as Spearman's Rank. Out of curiosity, I timed Kendall's tau against Spearman's rank on 80+ million rows. Spearman took 4.5 mins, and after 20+ minutes, I terminated Kendall's tau. Here's a replication with a smaller sample size (80k):

n = 80000
x <- rnorm(n = n)
y <- rnorm(n = n)
z <- rpois(n = n, lambda = 5)

test <- data.frame(z, x, y)



start <- Sys.time()
cor(x = test, method = "spearman")
end <- Sys.time()
end - start
#Time difference of 0.1559448 secs

start <- Sys.time()
cor(x = test, method = "kendall")
end <- Sys.time()
end - start
#Time difference of 8.224911 mins
#too damn long!

Is there some reason τ is less useful or meaningful with larger samples?

I don't think Kendall's tau loses meaning with larger data sets, it just takes too long to compute. I think if someone really wanted to use Kendall's tau, they could parallelize certain steps in the computation. Here's a discussion on the general computation: http://adereth.github.io/blog/2013/10/30/efficiently-computing-kendalls-tau/

Or is it just that τ is hard to compute and you might as well approximate it by randomly sampling pairs of points and checking how often they agree?

I'm usually against resampling. If data is under 20GB, in most cases, you can figure out how to run your computations without needing to resample, if time permits. However, if you're in a time crunch, then you can try a bootstrapped correlation with a limited number of runs. However, if your data set is large, this will have it's own computation issues if you do not parallelize the bootstrap runs.

However, if you don't really NEED Kendall's tau, why not use spearman's rank?

Answer 2

It is frustratingly unhelpful that the authors did not build more on this idea. One thing we can be certain of is that with large $n$, the supposed "assumption based" classical tests like a paired t-test, are robust and consistent estimators of mean difference. Regardless of the underlying distribution, with large $n$ ($n$ > 20 even) the paired t-test is a consistent estimator of mean difference which does NOT depend on the underlying distribution. This is simply a result of the well known central limit theorem.

It should be noted that rank based tests are also tests of mean differences when the distributions are symmetric. When they're not, I have no idea how to practically interpret Kendall's Tau in a manner which generalizes to a population of interest. Thus the $p$-value from both tests will either go to 0 if the null hypothesis is false or 1 if it is true in either setting.

That null hypothesis is, in most practical scenarios, whether the mean difference $d=0$. This is shared between test of Kendall's Tau and the paired t test. Kendall's Tau is widely misinterpretted as "non-parametric" in the sense that the underlying hypothesis is whether $\mathcal{H}_0: F_X = F_Y$ with $(x_i, y_i), i=1, \ldots, n$ being the paired dataset in question. This is patently false: it is neither the definition of a nonparametric test, nor is it a hypothesis tested by the Kendall's Tau. The correct unpaired test of this "strong null" hypothesis would be the Kolmogorov Smirnov, and I'm not aware of any paired analogue. You can plainly see this phenomenon with some toy examples: