I am working with a seven item Likert Scale (1, 2......6, 7) for a classroom management.

How can I calculate the distance between these seven scale points? This is for instrument reliability purposes, I want to see if people find, for example, a larger difference between choice 4 and 5 and a shorter distance/difference between choice 5 and 6.

There is an IRT method to figure this out, but I am coming up blank on it.

Any/all help is much appreciated.

  • 2
    $\begingroup$ A little more information please. How many subjects do you have (IRT models can require a large number of subjects)? Is this truly a Likert scale or a 7-point partial credit rating scale? Rating scale efficacy is part of the reliability question, but reliability is well defined and can be estimated other ways. More clarity on what you wish to measure or understand would help. $\endgroup$ – doug.numbers Feb 4 '14 at 19:30
  • 1
    $\begingroup$ There are 743 teacher respondents. This is a true Likert Scale - I am trying to convince my colleagues that a 7 point scale is too long and we can narrow down the options to say, 5 points. But in order to do that, I would like to provide them information on how respondents view the different points on the scale. E.g., Is the difference between 3 and 4 greater/lesser/equal than say, 6 and 7? Hope I did better at clarifying this time around. $\endgroup$ – Emme Feb 4 '14 at 22:24

Based on your response, you could handle the problem with classical test theory or item response theory. The easiest and fastest way to do this would be to calculate a total score for each subject, then calculate the average total score for each category of the rating scale. Your data will have to be converted from wide to long format to accomplish this, but that is easy to do. If the data are somewhat normally distributed, then you could calculate the mean and standard error of the total scores awarded for each rating scale category. The means should step up as the as the categories increase, and there should be a statistically significant difference between the category means. If there is not a significant difference, then the raters cannot tell the difference between the categories and they should be collapsed. This is better with item response theory as the ordinal values for total scores will be converted to true linear measures and distances between values will be more meaningful. IRT also provides wonderful graphs for this analysis.

That being said, if your data are in long format, you can do comparative boxplots with whiskers and notches. Visually the median values should step from lower to higher, evenly across scale and the notches between the boxes should not overlap (the notches are 95% confidence intervals. Example here. Hope this helps.

Let me know if there is anything else.

| cite | improve this answer | |
  • $\begingroup$ This is really helpful! thank you Doug. Would you recommend an article or two ? $\endgroup$ – Emme Feb 8 '14 at 3:03
  • $\begingroup$ 1.Fisher Jr. WP. Rating Scale Instrument Quality Criteria. Rasch Meas Trans. 2007;21(1):1095. 2. Linacre J. Optimizing Rating Scale Category Effectiveness. J Appl Meas. 2002;3(1):85–106. There is also Best Test Design and Rasch Rating Scale Analysis, both by Benjamin Wright. You don't have to get too fancy if you stick with the boxplot approach. $\endgroup$ – doug.numbers Feb 8 '14 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.