# How to compare Likert scales with varying number of categories across time?

Let Year 1 be last year's data and Year 2 be this year's data.

Suppose that in Year 1, you had a likert scale that was 1-9 (Categorical/Ordinal) and that in Year 2, for the same question you had a likert scale that was 1-5 (Categorical/Ordinal).

What would be some of the things that you would try (if at all) to compare the two years worth of data?

What I've done so far:

• Compared distributions (shape, skew, and kurtosis, statistically equal)
• Rescaled 1-9 to 1-5 and the changes YoY in frequencies match logical expectations derived from industry news/events and qualitative research findings.

Note: This is not homework. It also may not have a definite answer. But, I need a hand!

-
why do you say Likert scale and then Categorial/Ordinal? Likert means interval scaled. Can you clarify this a little? –  Henrik Feb 2 '11 at 10:36
To be more specific, the title should be changed to Likert "item". On your second point, I think a lot of people would disagree as to whether or not a Likert item presents interval or ordinal data. For my question, it's an agreement scale, from strongly disagree to strongly agree. Each level of agreement being a "category" and the distance between being "ordinal". But let's not get tied up in semantics! –  Brandon Bertelsen Feb 2 '11 at 11:04
@Henrik @Brandon There were already some discussions, headed under the scales tag, about the nature and the way to treat Likert scale/item. –  chl Feb 2 '11 at 19:23

This is not a complete answer; just a few points:

• If you can administer both versions of the scale to a subsample, you could estimate what corresponding scores are on the two response formats. Then you could apply a conversion formula that is empirically justified. I can think of a number of ways of doing this. I'd be interested if anyone has an academic paper on best practice for doing this.

• If you do a simple rescaling (1 = 1; 2 = 3; 3 = 5; 4 = 7; 5 = 9), there is no guaranty that this is justifiable. As a broad statement (at least within my experience in organisational settings) changes in item wording and changes in scale options are likely to have a greater effect on responses than any actual change in the attribute of interest. At the very least you should check whether the scale anchors used are roughly equivalent across the two response formats.

-
As a note to your second comment. The anchors are the same as they were in the previous year of the survey. Essentially, the granularity of the scale was reduced. –  Brandon Bertelsen Feb 2 '11 at 9:16

[Technically you've got survey items, not Likert scales; the latter are fashioned from multiple items. See, for example, Paul Spector's Summated Rating Scale Construction {Sage}.]

The steps you take will need to depend on the audience for which you're reporting. If it's academic and rigorous, like a dissertation committee, you may face special challenges. If it's not, and if it's comfortable with the common 1-5 format, why not rescale to fit that and then report means and standard deviations (especially since shapes, skew, and kurtosis are no different from year to year. I presume the distributions are normal enough that means accurately express central tendency?).

-->Why am I treating your variables as interval-level ones? Purists may say that ordinal-level variables should not be reported via means or s.d. Well, your comments suggest, despite your use of "categorical/ordinal," that you are dealing with an ordinal level of measurement which you actually feel comfortable treating as interval-level. After all, why otherwise would you assess skewness or kurtosis. I'm guessing that your audience, too, will be ok with and will be able to relate to interval-level statistics such as means.

It sounds good that you have already explored the data graphically. If you want to go beyond assessing the magnitude of the difference and conduct an hypothesis test, why not do a T-test (independent or correlated, depending on your data) comparing the 1-5 scores pre and the 1-5 scores post, and yielding a confidence interval for the mean difference. Here I'm assuming you've got random samples from a population.

-
Yes, I realize I'm not "supposed" to be looking at some of these things for ordinal data, but really, it's the only tool I could think of to compare the two years. Really, I was looking at things that could compare the distributions. But, I guess testing means could be plausible - but a confidence interval may not necessarily include my mean as there have been many structural changes to the industry for which this question reviews YoY. –  Brandon Bertelsen Feb 2 '11 at 9:11