How to equate questionnaires with different response scales

Question

I'm trying to help out a not-for-profit organization and made an error that I'm trying to fix. They've been doing a non scientific survey of their Board of Directors for a number of years regarding board engagement. They asked me to help them convert the survey format from an unfriendly Excel file to something web based, handle the results and do some analysis for them.

In past years, they've used what I'm going to call a weighted scale (Very good = 5, Good = 3, Fair = 2, Poor = -1). This year, we mistakenly included an extra level by allowing a 5, 4, 3, 2, or 1 response. (That is, the previous questionnaires had 4 options, but this one has 5.)

I'm trying to figure out how to conform the scales to be roughly equivalent. I had been thinking that I could let 5 = 5, 4 = 3, 3 = 2.5, 2 = 2 and 1 = -1. However, I realize that this doesn't work and will skew everything lower.

Can anyone recommend a good way to handle this?

Answer 1

This probably isn't the answer you were hoping for, but I'm not sure you should merge these. Presenting respondents with different possible answers in a survey will in and of itself affect their choices. Perhaps it's just because I come from the stricter area of psychometrics and behavioral research, but I don't think your conclusions would be valid. If you wanted to try this anyway, and wanted to be able to support your approach with an analysis showing that this merge wasn't affecting the answers you were receiving, you could create a survey with the merged numbers to reassess a randomly-sampled subset of the group you sent the second version of your survey to. Then, you could show that there was no statistically-significant difference between their new answers and the merged values you would have assigned them.