I have a dataset of 80 samples, with each containing two measurements on a 0-10 ordinal scale (
Rating_2) as well as their true values on the same scale (
Rating_True). I'm now looking for a way to visualize the deviation of each rater from the truth, allowing me to judge if raters tend to over- or underestimate the value.
Example dataset in R:
df <- structure(list(ID = 1:20, Rating_1 = c(9,8,6,9,8,6,7,3,9,4,5,5,0,1,6,3,2,1,0,7), Rating_2 = c(7,10,10,10,10,8,5,9,7,6,8,6,3,4,7,2,0,3,1,2), Rating_True = c(10,10,9,9,8,8,7,7,6,6,5,5,4,4,3,3,2,2,1,0)), class = "data.frame", row.names = c(NA, -20L))
Another option would be comparing BoxPlots of both raters for each level of the scale (example below).
Would you judge one of the approaches (or a unmentioned third approach) superior? If so, why? Could you point me to additional resources for finding a good approach?
Thanks a lot!