Which inter-rater reliability methods are most appropriate for ordinal or interval data? I believe that "Joint probability of agreement" or "Kappa" are designed for nominal data. Whilst "Pearson" and ...
I am reading a data mining book and it mentioned the Kappa statistic as a means for evaluating the prediction performance of classifiers. However, I just can't understand this. I also checked ...
I'm having a bit of trouble wrapping my head around something. I have a data set which contains two columns that essentially attempt to measure the same thing, one on a 1-50 continuous scale and ...
Much has been written on the ICC and Kappa, but there seems to be disagreement on the best measures to consider. My purpose is to identify some measure which shows whether there was agreement between ...
I have a decision support system that recommends Treatments (T1 to T8) to patients, where ...
Assessing and testing inter-rater agreement with kappa statistic on a set of binary and Likert items?
I am trying to calculate inter-rater reliability scores for 10 survey questions (on which there were 2 raters)--seven questions are binary (yes/no), and 3 are Likert-scale questions. Should ...
What are the tests to measure agreement between two methods of measurement when the unit of measurement is different between the two methods?
Bland-Altman plot measure the agreement between two different methods which measures the same variable. As far as I understand Bland-Altman can measure only if the two methods have the same unit of ...