In the course of my work, something of a problem has arisen. It can be summarized like this: there's a large pool of job applicants, let's say N of them. What needs to be done is that these applicants be ranked in order of desirability to the company: 1, 2, 3,... N. Until now, this has been done rather laboriously by discussions (read: arguments) between multiple people responsible for the decision. The idea is NOT to eliminate said discussions, but to reduce the amount of work necessary.
So, the new idea being batted about is that one could introduce a sort of "score" for each applicant... For the sake of argument, let's say that each applicant was rated from a low of 1 to a high of 5 for each of several different dimensions (e.g., raw intelligence, sociability, etc.), and the ratings were simply added up to generate a final score. Then the applicants could be ordered by score. Of course, there will be many many ties with this particular system, but it would reduce the problem of ranking each applicant by creating "bins" that could then be sorted through individually, i.e., independent of those outside a given bin.
But how to convince people that the "score" had any correlation with rankings as done by the old, laborious method? It seems to me that one of the common nonparametric rank correlation coefficients -- Spearman rho, Kendall tau -- should be suited to this purpose. (In particular, Kendall tau-b, as it is designed to handle ties.)
But I am not a statistician, and I wonder if something more appropriate exists. After all, using a nonparametric statistical test ignores the fact that the one of the variables being compared is simply the list (1, 2, 3,..., N)! Or perhaps my understanding of what "nonparametric" means in this context is flawed. Any insight into what methods I can use to attack this problem would be appreciated.
