Consider a problem where we ask a number of people to select and rank their top three choices out of a number of options. The set of options is the same for everyone, and they all have to rank their top three choices.

How can I go about the estimating the ranking of the options given data collected? How can I model the problem?

For example, say that we get the following data (only showing the first four options, and the rankings whenever someone included them in the ranking):

Option A: 1
Option B: 1,3,1,2,1
Option C: 3,3
Option D: 2,2,2

Where the way to read the above is:

  • First row: Only one person chose A as their top choice
  • Second row: Three people chose B as their top choice, one person chose it as their second best choice and one more person chose it as its third choice.

A trivial, but unhelpful answer would be to average the ranking for each option. That would be:

Option A: 1
Option B: (1+3+1+2+1)/5 = 1.6
Option C: (3+3)/2 = 3
Option D: (2+2+2)/3 = 2

The above averaging is unhelpful because it would not consider the number of people that chose a given option, and given the way the experiment is designed, the more often a given option is chosen in the top 3 list, the higher the ranking should be.

How can I go about modeling this problem?


A simple approach would be to assume the average of the ranked choices.

For example, suppose you have six options, ABCDEF. Suppose Joe picks A, D, and C as their first, second and third choice.

Then if there was a full ranking, B, E and F would occupy positions 4,5,6 in some unknown order. So why not pretend they all have the average of those ranks, i.e. 5, for Joe?


I think that the following three broad approaches might be helpful for this scenario. If I understand the scenario correctly, I would call this task a statistical modeling of surveys with partial data. The three approaches that I would consider are (please correct me, if I wrong - I'm learning statistics):

  • Cross-validation techniques (for a comprehensive description, see this paper);
  • Survey weighting (for details, please see this book and this research paper);
  • Missing values imputation (that is to re-formulate the problem as a missing data problem).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.