I am calculating inter-rater agreement for ~350 raters, each of who rated 100 items using Likert scale. The original scale has range [-3,3], using this python implementation, I got $\alpha=.35$, which is lower than I expected. Then I restricted the range of values to [-1,1] by clipping and got $\alpha=.29$ which is even lower. Should not clipping increase the agreement? Or I am not understanding it correctly?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.