# Max-Diff Using Rank-Order Data?

Many discussions of acquiring data for Max-Diff describe the following steps.

• Show all items and ask the respondent to pick the best
• Remove the item identified as the best, and ask respondents to pick the best item from those items remaining
• Repeat until all items but one have been selected

Would a standard rank-order question not produce the identical results?

Thanks in advance to all for any info.

One would think so, wouldn't one?

But, probably not. While mathematically the operations are the same, psychologically they are not. When asked to rank (say) 8 items, some people may go top-down, some bottom-up and some do something else. Ranking items from bottom-up is not the same as ranking them from top-down.

I don't have a link for this, but I bet it's been studied.

Peter Flom rightly points out that psychologically some respondents will start from their bottom preference and move upwards if asked to rank things, and this will be subtley different to the procedure of being asked the favourite of a shrinking list.

In addition and relatedly, there's a fairly well known phenomenon in experimental economics that preferences depend on the context provided. So (made up example), if asked to choose between paying:

• affordable rent increase for a nicer house or
• paying a very small fine,

people are more likely to rank the fine above the rent increase than if they are given a choice between:

• paying unaffordable amounts for a mansion,
• paying the affordable amount for a nicer house, and
• the small fine.

The context of thinking about the mansion, even though it is out of reach, makes the affordable but nicer house look more like the "middle" path.

Are people inconsistent enough for this to apply even when you have only just taken away the additional option five seconds before? Absolutely. Does this make one method more correct than the other? No. Should the results be pretty much the same? I would imagine so, but I'm sure they're not exactly the same. Whether this matters, and which one to choose, will depend on context.

I was trying to make the point that MaxDiff was a partial ranking task and did not yield any information that could not be obtained by a complete rank ordering of all the attributes. I was trying to explain why the claim that MaxDiff yielded ratio-scale estimates could not be supported. I used the ranking task that you described as a way to get a complete ranking that seemed easier than multiple best-worst selections as in MaxDiff. I did not advocate ranking as a measurement technique. In fact, I noted how rankings cannot be analyzed without some type of log-ratio transformation because they are ipsative.

However, I agree that it makes a difference whether you rank from best to worst or worst to best. It makes a difference whether you present the attributes in a blocks of three or four or five in a MaxDiff task. It makes a difference when you give pairs of attributes which comes first and which comes second. That is why we randomize the order in pairwise comparisons. Similarity judgments are not symmetric. Remember Tversky's example: Cuba is more similar to Russia than Russia is similar to Cuba. That is, dominant and subordinate members of a pair have different similarity when presented first and second. Preference is not transitive. It is not unusual to find that A is preferred over B and B is preferred over C, yet C is preferred to A. Clearly, context is important. Questions create their own microclimates.