2
$\begingroup$

Suppose I have a dataset with the following information:

  • N objects, each of which can be rated
  • The number of ratings per object is variable (raters are unidentified), e.g., there are 30 objects each with a unique id
  • Each row has the predicted rating, the actual rating, and the difference between the two, which is an ordinal value, say between 0 and 10

How do I measure the consistency of ratings per object and for the whole dataset? Basically, there is a distribution of "differences" per object. What are the various ways to approach this problem? Thank you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.