# Name and origin of a metric for evaluating ranking performance

I came across an evaluation metric to test whether a predicted rank is good, especially the top k items. But I don't know what it is called or where it is used, which makes my discussion of this metric very limited.

Suppose we have 5 items, A, B, C, D, and E.

Their corresponding values leading to the above rank are [ 10, 9, 7, 6, 3]. If k = 2, then our top k items are A and B.

Now without knowing the true rank or values, I trained a model and got a prediction for the ranking - [A, C, B, E, D]. Its top k items are A and C.

At the same time, I have another set of predictions from another method - [A, D, B, C, E]. Its top k items are A and D.

Although both have a top k accuracy of 0.5 because they both predicted Item A correctly only, the second one makes a worse decision on deciding the top k items.

To evaluate this, I first calculate the sum of the true values top k items, i.e. (A -> 10 + B -> 9) = 19.

For the first prediction, as A and C are chosen, the same calculation goes: (A -> 10 + B -> 7) = 17.

For the second prediction, the calculation is: (A -> 10 + D -> 6) = 16.

Then I divide the predicted top k sum by the true top k sum. So the first prediction got a value of 0.895. The second one got a value of 0.842. Therefore, the second prediction is worse than the first.

So does anyone know where does this method come from, and if I am using it correctly? My friend who works on active learning stuff told me this. But he can't remember where did he see this...