30
$\begingroup$

In my work, we are comparing predicted rankings versus true rankings for some sets of data. Up until recently, we've been using Kendall-Tau alone. A group working on a similar project suggested we try to use the Goodman-Kruskal Gamma instead, and that they preferred it. I was wondering what the differences between the different rank correlation algorithms were.

The best I've found was this answer, which claims Spearman is used in place of usual linear correlations, and that Kendall-Tau is less direct and more closely resembles Goodman-Kruskal Gamma. The data I'm working with doesn't seem to have any obvious linear correlations, and the data is heavily skewed and non-normal.

Also, Spearman generally reports higher correlation than Kendall-Tau for our data, and I was wondering what that says about the data specifically. I'm not a statistician, so some of the papers I'm reading on these things just seem like jargon to me, sorry.

$\endgroup$
  • 3
    $\begingroup$ "Spearman generally reports better correlation than Kendall-Tau for our data, and I was wondering what that says about the data specifically" ... likely nothing; Kendall $\tau$ is often nearer 0 than Spearman's $\rho$ when the correlations are not really close to $0$ or $\pm 1$ - it measures association differently; the fact that it's typically smaller in magnitude doesn't mean that the Spearman correlation is 'better'; they're just measuring different things about the data. What would lead you to say 'better correlation'? $\endgroup$ – Glen_b Nov 2 '13 at 20:48
  • 1
    $\begingroup$ That was indirectly the same as my question, @Glen_b; except, I was asking why the algorithms reported higher correlation and what would cause that. I'll change "better" to "higher" so to make my meaning a little more clear. You are right that they measure different things, and that the numbers don't really have too much to do with each other, but I wanted to know what the numbers actually meant, which is answered in detail below. $\endgroup$ – Poik Aug 25 '14 at 14:51
28
$\begingroup$

Spearman rho vs Kendall tau. These two are so much computationally different that you cannot directly compare their magnitudes. Spearman is usually higher by 1/4 to 1/3 and this makes one incorrectly conclude that Spearman is "better" for a particular dataset. The difference between rho and tau is in their ideology, proportion-of-variance for rho and probability for tau. Rho is a usual Pearson r applied for ranked data, and like r, is more sensitive to points with large moments (that is, deviations from cloud centre) than to points with small moments. Therefore rho is quite sensitive to the shape of the cloud after the ranking done: the coefficient for an oblong rhombic cloud will be higher than the coefficient for an oblong dumbbelled cloud (because sharp edges of the first are large moments). Tau is an extension of Gamma and is equally sensitive to all the data points, so it is less sensitive to peculiarities in shape of the ranked cloud. Tau is more "general" than rho, for rho is warranted only when you believe the underlying (model, or functional in population) relationship between the variables is strictly monotonic. While Tau allows for nonmonotonic underlying curve and measures which monotonic "trend", positive or negative, prevails there overall. Rho is comparable with r in magnitude; tau is not.

Kendall tau as Gamma. Tau is just a standardized form of Gamma. Several related measures all have numerator $P-Q$ but differ in normalizing denominator:

  • Gamma: $P+Q$
  • Somers' D("x dependent"): $P+Q+T_x$
  • Somers' D("y dependent"): $P+Q+T_y$
  • Somers' D("symmetric"): arithmetic mean of the above two
  • Kendall's Tau-b corr. (most suitable for square tables): geometric mean of those two
  • Kendall's Tau-c corr. (most suitable for rectangular tables): $N^2(k-1)/(2k)$
  • Kendall's Tau-a corr. (makes nо adjustment for ties): $N(N-1)/2 = P+Q+T_x+T_y+T_{xy}$

where $P$ - number of pairs of observations with "concordance", $Q$ - with "inversion"; $T_x$ - number of ties by variable X, $T_y$ - by variable Y, $T_{xy}$ – by both variables; $N$ - number of observations, $k$ - number of distinct values in that variable where this number is less.

Thus, tau is directly comparable in theory and magnitude with Gamma. Rho is directly comparable in theory and magnitude with Pearson $r$. Nick Stauner's nice answer here tells how it is possible to compare rho and tau indirectly.

See also about tau and rho.

$\endgroup$
14
$\begingroup$

Here's a quote from Andrew Gilpin (1993) advocating Maurice Kendall's $τ$ over Spearman's $ρ$ for theoretical reasons:

[Kendall's $τ$] approaches a normal distribution more rapidly than $ρ$, as $N$, the sample size, increases; and $τ$ is also more tractable mathematically, particularly when ties are present.

I can't add much about Goodman-Kruskal $γ$, other than that it seems to produce ever-so-slightly larger estimates than Kendall's $τ$ in a sample of survey data I've been working with lately... and of course, noticeably lower estimates than Spearman's $ρ$. However, I also tried calculating a couple partial $γ$ estimates (Foraita & Sobotka, 2012), and those came out closer to the partial $ρ$ than the partial $τ$... It took a fair amount of processing time though, so I'll leave the simulation tests or mathematical comparisons to someone else... (who would know how to do them...)

As ttnphns implies, you can't conclude that your $ρ$ estimates are better than your $τ$ estimates by magnitude alone, because their scales differ (even though the limits don't). Gilpin cites Kendall (1962) as describing the ratio of $ρ$ to $τ$ to be roughly 1.5 over most of the range of values. They get closer gradually as their magnitudes increase, so as both approach 1 (or -1), the difference becomes infinitesimal. Gilpin gives a nice big table of equivalent values of $ρ$, $r$, $r^2$, d, and $Z_r$ out to the third digit for $τ$ at every increment of .01 across its range, just like you'd expect to see inside the cover of an intro stats textbook. He based those values on Kendall's specific formulas, which are as follows: $$ \begin{aligned} r &= \sin\bigg(\tau\cdot\frac \pi 2 \bigg) \\ \rho &= \frac 6 \pi \bigg(\tau\cdot\arcsin \bigg(\frac{\sin(\tau\cdot\frac \pi 2)} 2 \bigg)\bigg) \end{aligned} $$ (I simplified this formula for $ρ$ from the form in which Gilpin wrote, which was in terms of Pearson's $r$.)

Maybe it would make sense to convert your $τ$ into a $ρ$ and see how the computational change affects your effect size estimate. Seems that comparison would give some indication of the extent to which the problems that Spearman's $ρ$ is more sensitive to are present in your data, if at all. More direct methods surely exist for identifying each specific problem individually; my suggestion would produce more of a quick-and-dirty omnibus effect size for those problems. If there's no difference (after correcting for the difference in scale), then one might argue there's no need to look further for problems that only apply to $ρ$. If there's a substantial difference, then it's probably time to break out the magnifying lens to determine what's responsible.

I'm not sure how people usually report effect sizes when using Kendall's $τ$ (to the unfortunately limited extent that people worry about reporting effect sizes in general), but since it seems likely that unfamiliar readers would try to interpret it on the scale of Pearson's $r$, it might be wise to report both your $τ$ statistic and its effect size on the scale of $r$ using the above conversion formula...or at least point out the difference in scale and give a shout out to Gilpin for his handy conversion table.

References

Foraita, R., & Sobotka, F. (2012). Validation of graphical models. gmvalid Package, v1.23. The Comprehensive R Archive Network. URL: http://cran.r-project.org/web/packages/gmvalid/gmvalid.pdf

Gilpin, A. R. (1993). Table for conversion of Kendall's Tau to Spearman's Rho within the context measures of magnitude of effect for meta-analysis. Educational and Psychological Measurement, 53(1), 87-92.

Kendall, M. G. (1962). Rank correlation methods (3rd ed.). London: Griffin.

$\endgroup$
9
$\begingroup$

These are all good indexes of monotonic association. Spearman's $\rho$ is related to the probability of majority concordance among random triplets of observations, and $\tau$ (Kendall) and $\gamma$ (Goodman-Kruskal) are related to pairwise concordance. The main decision to make in choosing $\gamma$ vs. $\tau$ is whether you want to penalize for ties in $X$ and/or $Y$. $\gamma$ does not penalize for ties in either, so that a comparison of the predictive ability of $X_{1}$ and $X_{2}$ in predicting $Y$ will not reward one of the $X$s for being more continuous. This lack of reward makes it a bit inconsistent with model-based likelihood ratio tests. An $X$ that is heavily tied (say a binary $X$) can have high $\gamma$.

$\endgroup$
  • 2
    $\begingroup$ Frank, can you explain it Spearman's ρ is related to the probability of majority concordance among random triplets of observations in more details, not very mathematically hard, if possible? Thanks. $\endgroup$ – ttnphns Nov 3 '13 at 15:02
  • 1
    $\begingroup$ I read that many years ago, probably in a nonparametric statistics text. I have been unable to find the reference. $\endgroup$ – Frank Harrell Nov 4 '13 at 12:50
  • 1
    $\begingroup$ Unfortunate... :-( Because the statement itself is very intriguing. $\endgroup$ – ttnphns Nov 4 '13 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.