Consider the following survey question:

Q: How would you classify the importance for you of the following 5 items:


Assign to each item a number in the set {1,2,3,4,5}, with 1 meaning the highest importance and 5 meaning the lowest importance; the number used to an item cannot be used in any other item.

  • How can one test whether item A is the most important item for a sample with 100 individuals?

The naive approach would be to compute the marginal distribution of rankings (e.g., mean score for each item), but it would throw away a lot of information as it does not account for the within-person relationship between ranks.

As an extension to paired preference model (e.g., the Bradley-Terry model, described in Agresti's CDA pp. 436-439), there exist model for ordinal or likert-type comparison data with or without subject covariates, as well as model for ranking data (baiscally, it relies on the use of log-linear model). Here is a short intro to the package, and a mathematical explanation in this technical report: Fitting Paired Comparison Models in R. You will find everything you need in the prefmod R package, see the pattR.fit() function which expects data in the form you described:

 The responses have to be coded as consecutive integers starting
 with 1.  The value of 1 means highest rank according to the
 underlying scale.  Each column in the data file corresponds to one
 of the ranked objects. For example, if we have 3 objects denoted
 by ‘A’,‘B’,and ‘C’, with corresponding columns in the data matrix,
 the response pattern ‘(3,1,2)’ represents: object ‘B’ ranked
 highest, ‘C’ ranked second, and ‘A’ ranked lowest. Missing values
 are coded as ‘NA’, ties are not allowed (in that case use
 ‘pattL.fit’.  Rows with less than 2 ranked objects are removed
 from the fit and a message is printed.

For additional information (about and beyond your particular study), you might find useful the following papers:

  1. Böckenholt, U. and Dillon, W.R. (1997). Modelling within-subject dependencies in ordinal paired comparison data. Psychometrika, 62, p.411-434
  2. Dittrich, R., Francis, B., Hatzinger, R., and Katzenbeisser, W. (2006). Modelling dependency in multivariate paired comparisons: A log-linear approach. Mathematical Social Sciences, 52, 197-209.
  3. Maydeu-Olivares, A. (2004). Thurstone's Case V model: A structural equations modeling perspective. In K.van Montfort et al. (eds), Recent Developments on Structural Equation Models, 41-67.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.