How would you deal with lists of varying lengths? For example, if you asked for an ordered list of your favourite movies without any constraints how would you value the weight of someone who submits 5 samples and another who submits 20 samples?
Aside from the linear valuations, crudely you could give each participant's first selection a score of n, 2nd = n-1, 3rd = n-2... nth=1/nth value (n being the longest list supplied) and build an aggregated score across the sampled population. But that feels far too crude for proper analysis. Is there a better weighting distribution with a tapered decay to suit a long tail?
Instinctively, the person who supplies a longer submission should weigh higher than a shallow submission. But I'm unsure of the social research techniques that might be suitable.
Is there a statistical model that addresses this kind of analysis for open response lists?