# Kendals tau-b correlation with lots of 0 variance columns

I am writing a paper that is meant to establish that certain algorithms rank candidate values accross multiple problems in a different way. To do this, I take Algorithm A and run it on the candidate values and do the same for candidate B. When I run them, I notice that for multiple problems they rank the values as follows:

     A     B

c1   1     1
c2   2     1
c3   3     1


Is there a metric that can tell me something about how simillar these 2 ranking sets are without saying the correlation is undefined. fyi. I've tried Kendall's tau and it is undefined for the example given and so therefore I am looking for a metric that can tell me something useful in this particular case.

• Do you have a true ranking to which you can compare or are you more concerned with how different the rankings are apart from a known measure of accuracy? Commented Sep 23, 2019 at 3:55
• I am more concerned with how different the rankings are. Commented Sep 23, 2019 at 4:59
• Can your provide some pseudo-data. I am not sure I understand what a 0 column represents. Is Tau-b = 0 or all predictors zero for either or both of the candidate values? Commented Sep 23, 2019 at 22:05
• Edited and added an example. A 0 variance column is one where all the candidates are tied eg. the B column on the example given. Commented Sep 24, 2019 at 2:03
• I don’t know that there is a fix for this scenario. Can you calculate the index using something other than whole integers? Commented Sep 25, 2019 at 13:56

I found a solution. It's a metric called Wilsons e by Wilson(1970). See Gonzalez et al. (1996) for details

Bibtex below:

@article{gonzalez1996measuring,
title={Measuring ordinal association in situations that contain tied scores.},
author={Gonzalez, Richard and Nelson, Thomas O},
journal={Psychological bulletin},
volume={119},
number={1},
pages={159},
year={1996},
publisher={American Psychological Association}
}