What is the best way to analyze rank-ordered data when there are signs that respondents were less diligent/able to assign lower ranks? Is it sufficient to introduce a dummy for lower/earlier ranks into the model or should one model this directly?
I have a set of survey data containing rank-orderings. More specifically, a large sample of respondents was asked to indicate which groups of people should first get access to a COVID-19 vaccine once it becomes available. They ranked 7 different groups from highest to lowest priority; ties were not permitted, rankings had to be complete and the initial ordering of the groups was randomized across respondents.
In my analysis so far, I followed Allison & Christakis (1994, "Logit Models for Sets of Ranked Items", Sociological Methodology). Descriptive analyses show a quite consistent rank-ordering across a number of demographics and non-parametric tests (Friedman & Wilcoxon) indicate significant differences between pairs of choices.
The problem is: It turns out that respondents seem to have been less diligent in assigning the lower ranks, or at least less able to do so (not unrealistic, IMO). Following Allison/Christakis (pp. 216-218), I created a dummy for higher (1-4) and lower (5-7) ranks and included it as an individual-specific covariate into the model. The interactions between the dummy and the alternatives were significant and the coefficient estimates for later choices also show signs of greater randomness (the standard deviation between the coefficients for early choices is 3 times the SD for late choices).
My question is: What is the best way to deal with this? If I understand Allison/Christakis correctly, I should treat later choices as ties by, it seems to me, keeping the early/late dummy in the model - or, in
Stata at least, code all later choices as 0 and use the
incomplete() option for
One alternative is to use models that allow for unobserved heterogeneity in ranking capabilities (Fok. et al., "A Rank-Ordered Logit Model with Unobserved Heterogeneity in Ranking Capabilities", Journal of Applied Econometrics), but I am not aware where this would be implemented (not in Stata and the
gmnl package for R by Sarrias & Daziano cannot deal with ranked data, as far as I can tell). A final alternative, but hardly a great one, would be to use only the most-preferred choice as the outcome and use the regular conditional logit (or multinomial logit, as it is also known).
Any better suggestions or pointers to relevant packages would be greatly appreciated!