This may be a very stupid newbie question, but as the title says, I am looking for a rank correlation coefficient that takes into account the presence of different elements in two rankings. I have tried Spearman's ρ and Kendall's τ, but in both cases, when I calculate rank correlation between the following two vectors in R, I get a coefficient of 1.

var1 <- c("a","b","c")

var2 <- c("a","b","d")

What I'd be interested in is a value that also expresses the fact that element 3 in the two vectors is different. Maybe I'm missing the obvious solution or this is simply something that would need two metrics rather than a single one. Any hints would be appreciated.

EDIT: thanks to user mic I moved from correlation to similarity and found a bunch of useful papers; https://ragrawal.wordpress.com/2013/01/18/comparing-ranked-list/ has a great discussion and Python implementation of a particular measure called "rank biased overlap". Problem solved!

  • $\begingroup$ One of the benefit of rank correlation such as Spearman rho is to be robust to a monotonic transform of your data, and only relies on order. It is apparently what you are seeking. Why not considering another "correlation" or similarity measures? Or penalize the rank correlation with another index? $\endgroup$ – mic Jun 22 '15 at 16:38
  • 1
    $\begingroup$ I am bemused at how you can get Spearman correlation to be calculated for two vectors of characters (or whatever the precise jargon is in R). Could you guide us through how that makes sense? $\endgroup$ – Nick Cox Jun 22 '15 at 16:56
  • $\begingroup$ @NickCox spearman can be used for ordinal variables and the R implementation can derive rank from any variable type $\endgroup$ – Bernhard Jun 22 '15 at 17:12
  • 1
    $\begingroup$ @mic using a similarity measure is a great idea, should have thought about that, thanks! $\endgroup$ – Bernhard Jun 22 '15 at 17:13
  • 2
    $\begingroup$ OK, so the implication is that alphabetic (or string sort) order is sacrosanct in determining rank. I don't think that can mesh (easily) with your desire to recognise "different" as different characters could be the same rank, and vice versa. $\endgroup$ – Nick Cox Jun 22 '15 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.