4
$\begingroup$

The manual page for R's cor says:

Some people have noted that the code for Kendall's tau is slow for very large datasets (many more than 1000 cases). It rarely makes sense to do such a computation, but see function cor.fk in package pcaPP.

Why wouldn't it make sense to compute a Kendall's $\tau$ for a large sample? Is there some reason $\tau$ is less useful or meaningful with larger samples? Or is it just that $\tau$ is hard to compute and you might as well approximate it by randomly sampling pairs of points and checking how often they agree?

$\endgroup$
4
  • $\begingroup$ I wonder what is meant by "very large datasets". I mean, if you have 3.5 GB file with 50+ million rows, any correlation will take some time to compute. $\endgroup$
    – Jon
    Sep 20, 2016 at 18:31
  • $\begingroup$ @Jon The language "many more than 1,000 cases" suggests they mean on the order of 2,000, which doesn't seem like a "very large" dataset by modern standards. $\endgroup$ Sep 20, 2016 at 18:40
  • $\begingroup$ Old documentation? At one point, 1000 obs was considered a lot of data. Out of pure curiosity, I'm running Kendall's tau vs Spearman on 80+ million rows. So far, Kendall's tau is taking a loooong time; but Spearman is no rabbit either. $\endgroup$
    – Jon
    Sep 20, 2016 at 18:46
  • 1
    $\begingroup$ I can't mind read across space and time, but three possibilities are that (1) if you have a sufficiently large dataset, you should be trying something more ambitious any way (2) such a dataset is likely to be heterogeneous in some way that undermines the value of any single simple summary statistic (3) most of all, the question of whether there is a detectable relationship is moot for large sample sizes, while the magnitude of correlation isn't often of substantive interest otherwise. $\endgroup$
    – Nick Cox
    Sep 20, 2016 at 19:20

2 Answers 2

4
$\begingroup$

Here's my take on things:

Why wouldn't it make sense to compute a Kendall's τ for a large sample?

If time is really an issue, there are other options for nonparametric correlation statistics. From my understanding, Kendall's tau was meant for small samples (<100 observations). So why not use Spearman's rank?

SPSS documentation has some guidance for this: https://statistics.laerd.com/spss-tutorials/kendalls-tau-b-using-spss-statistics.php

In general, you may not want to compute Kendall's tau if time is a serious issue. There are other alternatives that have a lower computation time such as Spearman's Rank. Out of curiosity, I timed Kendall's tau against Spearman's rank on 80+ million rows. Spearman took 4.5 mins, and after 20+ minutes, I terminated Kendall's tau. Here's a replication with a smaller sample size (80k):

n = 80000
x <- rnorm(n = n)
y <- rnorm(n = n)
z <- rpois(n = n, lambda = 5)

test <- data.frame(z, x, y)



start <- Sys.time()
cor(x = test, method = "spearman")
end <- Sys.time()
end - start
#Time difference of 0.1559448 secs

start <- Sys.time()
cor(x = test, method = "kendall")
end <- Sys.time()
end - start
#Time difference of 8.224911 mins
#too damn long!

Is there some reason τ is less useful or meaningful with larger samples?

I don't think Kendall's tau loses meaning with larger data sets, it just takes too long to compute. I think if someone really wanted to use Kendall's tau, they could parallelize certain steps in the computation. Here's a discussion on the general computation: http://adereth.github.io/blog/2013/10/30/efficiently-computing-kendalls-tau/

Or is it just that τ is hard to compute and you might as well approximate it by randomly sampling pairs of points and checking how often they agree?

I'm usually against resampling. If data is under 20GB, in most cases, you can figure out how to run your computations without needing to resample, if time permits. However, if you're in a time crunch, then you can try a bootstrapped correlation with a limited number of runs. However, if your data set is large, this will have it's own computation issues if you do not parallelize the bootstrap runs.

However, if you don't really NEED Kendall's tau, why not use spearman's rank?

$\endgroup$
5
  • 1
    $\begingroup$ "why not use Spearman's rank?" — The appeal of the Kendall correlation, to me, is that it has a nice interpretation as an estimate of the probability of a randomly selected pair of points being concordant, rescaled from [0, 1] to [-1, 1]. $\endgroup$ Sep 20, 2016 at 19:54
  • $\begingroup$ From wikipedia: Contrary to the Spearman correlation, the Kendall correlation is not affected by how far from each other ranks are but only by whether the ranks between observations are equal or not, and is thus only appropriate for discrete variables but not defined for continuous variables. (en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient). Some limitations for Kendall. $\endgroup$
    – Jon
    Sep 20, 2016 at 20:25
  • 4
    $\begingroup$ "Not defined for continuous variables"? I don't think that follows, nor do I think it's true. $\endgroup$ Sep 20, 2016 at 20:31
  • $\begingroup$ SPSS documentation says: "Assumption #1: Your two variables should be measured on an ordinal or continuous scale."; However, minitab says: "Kendall's tau-b is a nonparametric measure of association for ordinal data. Ordinal data are categorical variables that have three or more levels with a natural ordering, such as strongly disagree, disagree, neutral, agree, and strongly agree. Kendall's tau-b is used in cross tabulation to measure the association between two ordinal variables." $\endgroup$
    – Jon
    Sep 21, 2016 at 15:52
  • 1
    $\begingroup$ Also, I took a look at some Stata documentation, and their docs seem to validate that Kendall's tau was meant for small samples (size not specified): "ktau displays Kendall’s rank correlation coefficients between the variables in varlist or, if varlist is not specified, for all the variables in the dataset. ktau is intended for use on small- and moderate-sized datasets; it requires considerable computation time for larger datasets." $\endgroup$
    – Jon
    Sep 21, 2016 at 15:53
0
$\begingroup$

It is frustratingly unhelpful that the authors did not build more on this idea. One thing we can be certain of is that with large $n$, the supposed "assumption based" classical tests like a paired t-test, are robust and consistent estimators of mean difference. Regardless of the underlying distribution, with large $n$ ($n$ > 20 even) the paired t-test is a consistent estimator of mean difference which does NOT depend on the underlying distribution. This is simply a result of the well known central limit theorem.

It should be noted that rank based tests are also tests of mean differences when the distributions are symmetric. When they're not, I have no idea how to practically interpret Kendall's Tau in a manner which generalizes to a population of interest. Thus the $p$-value from both tests will either go to 0 if the null hypothesis is false or 1 if it is true in either setting.

That null hypothesis is, in most practical scenarios, whether the mean difference $d=0$. This is shared between test of Kendall's Tau and the paired t test. Kendall's Tau is widely misinterpretted as "non-parametric" in the sense that the underlying hypothesis is whether $\mathcal{H}_0: F_X = F_Y$ with $(x_i, y_i), i=1, \ldots, n$ being the paired dataset in question. This is patently false: it is neither the definition of a nonparametric test, nor is it a hypothesis tested by the Kendall's Tau. The correct unpaired test of this "strong null" hypothesis would be the Kolmogorov Smirnov, and I'm not aware of any paired analogue. You can plainly see this phenomenon with some toy examples:

$\endgroup$
11
  • 1
    $\begingroup$ I don't understand your third paragraph. Providing the definition of "nonparametric" you have in mind might clear that up. $\endgroup$
    – whuber
    Sep 20, 2016 at 18:58
  • $\begingroup$ It's odd that you appear to take Kendall's tau as assessing difference in location (with difference in means presumed the case of most interest). It's a rank correlation and there is no expectation that distributions coincide. I have to suggest you're thinking of WIlcoxon-Mann-Whitney. $\endgroup$
    – Nick Cox
    Sep 20, 2016 at 18:58
  • 1
    $\begingroup$ You seem to be focusing on significance tests of correlations. The original statement isn't saying anything about significance testing. Also, $t$-tests are not robust (in fact, they are as far from robust as it's possible for a statistic to be), because the $t$-statistic has an asymptotic breakdown point of 0. $\endgroup$ Sep 20, 2016 at 19:05
  • 1
    $\begingroup$ @whuber a "non-parametric test" definition which embraces the $t$-test and test of Kendall's $\tau$ is as follows: a test is said to be non-parametric if a summary measure (or statistic) for a general distribution can be provided, e.g. $\theta = \int f(x) dF_X$ such that given any two (possibly different) DFs, the power of the test goes to 1 if and only if $\theta_1 \ne \theta_2$ for $\theta_1, \theta_2$ arising from the two DFs. $\endgroup$
    – AdamO
    Sep 20, 2016 at 19:22
  • $\begingroup$ @NickCox What does it mean for distributions to coincide? To have the same location? I understand it is a rank correlation, analogous to Pearson correlation in the sense that it is bounded by -1 and 1 indicating monotonic agreement rather than linear (as the Pearson cor. measures), The test of Pearson correlation is asymptotically equivalent to the t-test of a regression coefficient. I have also pointed out that rank location is a test of median when distributions are symmetric (which is also the mean except for Cauchy distributions). In other scenarios, rank location is tough to interpret. $\endgroup$
    – AdamO
    Sep 20, 2016 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.