0
$\begingroup$

I have n measured rankings for k objects, which I want to combine such that the combined ranking maximizes the average correlation with the measured rankings. As correlation measures I need solutions for spearman's rank correlation and for Kendall-Taus.

The intention here is to find the upper limit on the average correlation for the given data to compare to the performance of various models & predictors.

I have searched through this webpage, wikipedia and some forums, but found only references to heuristic methods like averaging or taking medians etc. and many posts describing that you might come to different conclusions if you weigh the different rankings differently.

While this is all true, it does not solve my problem. So to emphasize the points how my question differs: 1) There is no weighting or different interpretation for the different rankings (think: I asked n people to rank these objects, all opinions are equally important) 2) The metric for what is the best ranking is clear: Maximize spearman rank correlation/ Kendall-taus. I am not searching for just any method which sensibly averages rankings.

I would be grateful for any hints to literature on how to do this!

P.S: I guess for spearman simply averaging the ranks should do the trick, but it would still be great to have a source confirming this (or tell me if it should be the median instead for example)

Improve this question
$\endgroup$
2
  • $\begingroup$ Many related posts $\endgroup$
    – kjetil b halvorsen ♦
    Feb 19, 2020 at 15:03
  • $\begingroup$ well, yes, but I could not find one which answers my specific question and I still don't know an answer $\endgroup$
    – Xenon
    Feb 19, 2020 at 15:47

1 Answer 1

Reset to default
2
$\begingroup$

You are correct that simply averaging the ranks for each of the $k$ objects will provide you the overall rank that maximises the Spearman $\rho$ correlation with each of the individual ranks. This is equivalent to the Borda count method. The reference for this is Kendall & Gibbons (1990, pg. 125).

Finding the ranking that maximises Kendall's $\tau$ is a little more complicated. The Kemeny-Young method can determine the overall rank that minimises the Kendall's $\tau_D$ rank distance between itself and each of the individual ranks. Because Kendall's $\tau$ is a normalisation of the $\tau_D$ distance between two ranks,

$$\tau=1-2 \binom k2 ^{-1} \tau_D$$

The Kemeny-Young method yields the overall rank with the highest Kendall's $\tau$ correlation. Note this is problem is NP-hard though, so if you're working with large $k$, computation may be challenging.

For ways to calculate the average $\rho$ and $\tau$ between a criterion (e.g. overall) ranking and a group of ranks, see Lyerly (1952) and Hays (1960) respectively.

References

Hays, W. L. (1960). A note on average tau as a measure of concordance. Journal of the American Statistical Association, 55(290), 331-341.

Kendall, M. G. and J. D. Gibbons. 1990. Rank Correlation Methods. 5th ed. London: Griffin.

Lyerly, S. B. (1952). The average Spearman rank correlation coefficient. Psychometrika, 17(4), 421-428.

Improve this answer
$\endgroup$
2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.