# How to reduce number of items on a multi-item scale where each item requires ranking four response options

### Context

I have a survey of 16 questions, each with four possible responses. The purpose of the survey is to measure the respondent's propensity towards four categories (which we will denote A, B, C, D). Each of the four responses per question are representative of an aspect of the four categories, A, B, C, D.

The respondent rank orders each of the four responses (we will denote the first response by "4", the second by "3", etc).

To score the categories, we add the responses up based on the coding above. There are 16 x (4 + 3 + 2 + 1) = 160 total points. The sums for each category are computed, and the maximum score is deemed the respondent's dominant category.

Therefore each survey looks like the following (in CSV format)

question_num, A, B, C, D
1, 4, 3, 1, 2
2, 3, 4, 1, 2
3, 3, 4, 2, 1
4, 4, 3, 1, 2
5, 4, 3, 1, 2
6, 4, 3, 2, 1
7, 4, 3, 1, 2
.
.
.
16, 3, 4, 1, 2
sums, 64, 48, 24, 24


I have about 325 surveys completed.

### Aim

I want to remove possible redundant items in the survey so I can reduce the burden on future respondents.

### Questions

• My first strategy was to do a multi-logistic regression with the response as the dominant category (described above). Is this a good idea?
• Are there any other strategies for identifying redundant items?
• How are you defining redundancy? – user28 Jul 29 '10 at 4:15
• Maybe item-response theory? – Manoel Galdino Jun 8 '11 at 21:55

So restructure your data merging all user responses, so in such form:

Q1 Q2 Q3 ...
user1 rank for option1 for Q1, user1 rank for option1 for Q2, ...
user1 rank for option2, ...
...
user2 rank for option1, ...
...
user325 rank for option4, ...


And then cluster the questions. I recommend agglomerative clustering, there it is easy to see what questions can be removed.

I've never had to perform such analyses, but there is an academic literature on the factor analysis of ipsative tests that would be relevant: e.g.,

You could estimate a Thurstonian IRT model and use factor loadings and utilities as criteria to assess an items quality. With Brown & Maydeu-Olivares seminal work, it is possible to free ranking data from restrictions due to ipsativity.

A Thurstonian IRT model will help you identifying weaker questions (through item loadings and item utilities) which may indicate which question you can leave out.

Here is how to implement this approach using Mplus. There is also a useful implementation for R: package kcirt.