# Ranking estimation with partial data

Consider a problem where we ask a number of people to select and rank their top three choices out of a number of options. The set of options is the same for everyone, and they all have to rank their top three choices.

How can I go about the estimating the ranking of the options given data collected? How can I model the problem?

For example, say that we get the following data (only showing the first four options, and the rankings whenever someone included them in the ranking):

Option A: 1
Option B: 1,3,1,2,1
Option C: 3,3
Option D: 2,2,2


Where the way to read the above is:

• First row: Only one person chose A as their top choice
• Second row: Three people chose B as their top choice, one person chose it as their second best choice and one more person chose it as its third choice.

A trivial, but unhelpful answer would be to average the ranking for each option. That would be:

Option A: 1
Option B: (1+3+1+2+1)/5 = 1.6
Option C: (3+3)/2 = 3
Option D: (2+2+2)/3 = 2


The above averaging is unhelpful because it would not consider the number of people that chose a given option, and given the way the experiment is designed, the more often a given option is chosen in the top 3 list, the higher the ranking should be.

How can I go about modeling this problem?

## 2 Answers

A simple approach would be to assume the average of the ranked choices.

For example, suppose you have six options, ABCDEF. Suppose Joe picks A, D, and C as their first, second and third choice.

Then if there was a full ranking, B, E and F would occupy positions 4,5,6 in some unknown order. So why not pretend they all have the average of those ranks, i.e. 5, for Joe?

I think that the following three broad approaches might be helpful for this scenario. If I understand the scenario correctly, I would call this task a statistical modeling of surveys with partial data. The three approaches that I would consider are (please correct me, if I wrong - I'm learning statistics):

• Cross-validation techniques (for a comprehensive description, see this paper);
• Survey weighting (for details, please see this book and this research paper);
• Missing values imputation (that is to re-formulate the problem as a missing data problem).