I have a dataset of 80 samples, with each containing two measurements on a 0-10 ordinal scale (Rating_1
and Rating_2
) as well as their true values on the same scale (Rating_True
). I'm now looking for a way to visualize the deviation of each rater from the truth, allowing me to judge if raters tend to over- or underestimate the value.
Example dataset in R:
df <- structure(list(ID = 1:20,
Rating_1 = c(9,8,6,9,8,6,7,3,9,4,5,5,0,1,6,3,2,1,0,7),
Rating_2 = c(7,10,10,10,10,8,5,9,7,6,8,6,3,4,7,2,0,3,1,2),
Rating_True = c(10,10,9,9,8,8,7,7,6,6,5,5,4,4,3,3,2,2,1,0)),
class = "data.frame", row.names = c(NA,
-20L))
I was thinking about Bland-Altman plots, comparing the mean deviation of each of the two raters to the truth separately (example below).
Another option would be comparing BoxPlots of both raters for each level of the scale (example below).
Would you judge one of the approaches (or a unmentioned third approach) superior? If so, why? Could you point me to additional resources for finding a good approach?
Thanks a lot!