Why is Kendall's tau not consistent?

Question

Kendall's tau measures the concordance between the two observations. For example,

for $(X_1, Y_1)$, and $(X_2, Y_2)$,

the pair is said to be concordant if

$X_1 > X_2$ and $Y_1 > Y_2$, or

$X_1 < X_2$ and $Y_1 < Y_2$,

otherwise, they are discordant. Hene, Kendall's tau is calculated as the difference between the probability of concordance and the probability of discordance.

I read many articles about the consistency of Kendall's tau. I find that Kendall's tau is not consistent. That is, Kendall's tau may indicate that the two variables are independent while they are not. I really do not understand that because since Kendall's tau give us the average of the concordance and discordance, then It should be consistent.

Here is one example of what I have read:

From the abstract of Bergsma and Dassion, 2014[1]:

"The most popular ways to test for independence of two ordinal random variables are by means of Kendall’s tau and Spearman’s rho. However, such tests are not consistent, only having power for alternatives with “monotonic” association."

and from the introduction of the same source:

"A drawback for certain applications is that τ and ρS may be zero even if there is an association between X and Y, so tests based on them are inconsistent for the alternative of a general association."

Here the authors said that Kendall's tau and Spearman's rho may be zero even if there is an association! I did not really understand this point.

My question is, why is Kendall's tau not consistent?

Help with an example would be much more appreciated.

[1] Bergsma, W. and Dassios, A. (2014),
"A consistent test of independence based on a sign covariance related to Kendall’s tau"
Bernoulli 20(2), 1006–1028
https://projecteuclid.org/download/pdfview_1/euclid.bj/1393594014
(also at https://arxiv.org/pdf/1007.4259.pdf)

Answer 1

This is an issue of a measure (and hence the test based on it) not being able to pick up an association (an alternative under the test) it's not designed for.

In the same sense a t-test of means is not "consistent" against a difference in distributions that only related to a change in the spread. Almost any test or statistic designed to measure something will suffer from this kind of "deficiency" of failing to detect something different from that.

The Kendall and Spearman correlations do indeed measure monotonic association; they're fine when that's what you're interested in. However they don't attempt to measure every kind of association (and it would be a mistake to treat them as if they were).

For example, if $X$ is standard normal, then the Spearman, Kendall and Pearson correlation of $X$ with $|X|$ is $0$, even though if you know $X$ you know $|X|$ (there's a function dependence there). They are associated but the relationship is not monotonic (let alone linear).

The plot on the left shows perfect linear association. The Pearson, Spearman and Kendall correlations would all be 1. The plot in the center shows perfect monotonic association (when one variable increases, so does the other). The Spearman and Kendall correlations are still 1, but the Pearson correlation (which measures linear association) will be less than 1. The plot on the right shows a more general form of association, one that is non-monotonic. The sample Pearson, Spearman and Kendall correlations would all be essentially zero (the population measures will be exactly zero, by design).

Several additional illustrations of the third case (for less then perfect functional association) can be seen among the third row of plots here (specifically, the first and fourth item in that row):

This plot is from the Wikipedia page on Correlation and dependence, image is by Denis Boigelot, placed in the public domain.

We should not blame these measures for failing to do what they were not designed to do -- and in that sense, calling them inconsistent is potentially misplaced. In the sense it's being used in your quotes (outside what they're designed to do), in general almost any measure will be inconsistent.

[On the other hand, if the purpose of these sections is just to point out not to use these measures of association (or the corresponding tests) inappropriately, this is all to the good -- but then there seems little need to frame the issue this way.]