I work for an organization that oversees around 180 schools across the country. We regularly collect outcome data from these schools and rank them. I have been asked to do a survey of the national office (around 100 people) to determine what the best and worst perceived schools are, based on the staff's non-data based impression of school quality.
Some additional information:
- Not all staff members are familiar with all schools. When asked to rank a school "I don't know anything about this school" needs to be an option.
- Some staff work with a specific subset of schools, so these staff member's knowledge is great for these specific schools.
My first approach would be to randomly generate sets of 10 or so schools, with each school being in 6 or so sets. Then ask each staff member to rank each set, indicating which schools are unknown. I could then average the ranking for each school across it's sets and use that as the global ranking.
Is the above described method viable? Is there a better way? If this way would work, how many data points do I need for each school for the aggregated ranking to be representative?