I'm afraid I'm not answering your question exactly, but I am aware of a similar method called Q-sorting. This involves sorting items into a forced distribution according to predefined groups, such as,
preferred, and sometimes repeating this sorting process for the middle group to divide it into three intermediate groups (e.g.,
less). Typically, I think q-sorting forces answers into a quasi-normal distribution, so that the most items get sorted into the middle group, and the fewest items get sorted into the two extreme groups, but the number of items sorted into each group on either side of the middle group is symmetrical (e.g., same number of items sorted into both the
The Riverside Accuracy Project (RAP) has a web page with applied examples of a Q-sorter program that you might find helpful. I imagine others are available as well, and hope to see other answers here! I'm interested in these methods myself, and have designed my own measure of values that forces 25 items into 5 groups of 5 each according to their importance. That is, my own application is different from the quasi-normal forced distribution in that it forces a flat distribution instead.
The basic approach of presenting a limited, randomized subset of items for comparison to one another is not one I used, but I think it's used in the Q-sorting program at the RAP page above. Since different methods don't necessarily force answers into groups of the same or pre-specified sizes, I don't see why you couldn't just vary the method to achieve rankings instead of groupings. Maybe this wouldn't be Q-sorting per se though.