0
votes
0answers
17 views

Kappa near to 60% in unbalanced (1:10) data set

Copying my question from Data Science SE, where I haven't recieved answer: http://datascience.stackexchange.com/questions/1108/kappa-near-to-60-in-unbalanced-110-data-set. I have a classification ...
1
vote
0answers
23 views

What are the tests to measure agreement between two methods of measurement when the unit of measurement is different between the two methods?

Bland-Altman plot measure the agreement between two different methods which measures the same variable. As far as I understand Bland-Altman can measure only if the two methods have the same unit of ...
0
votes
0answers
141 views

Agreement statistics for confusion matrix with nominal categories

I have a decision support system that recommends Treatments (T1 to T8) to patients, where ...
1
vote
1answer
531 views

How to calculate error matrices, K hat and var(K hat)?

I have a very simple data set consisting of three columns: Ground Truth Canopy Class, Method 1 Canopy Class and Method 2 Canopy Class. Each row in the columns represents the canopy class (i.e. 1 ...
1
vote
0answers
268 views

How should I interpret the Kappa Fleiss test?

I noticed that when I have tables which the values are only 0 and 1, I get a kappa of 1 when the table is completely full of one, and when I have a table of zeros I get NaN as result using the irr ...
4
votes
1answer
120 views

Ordered category agreement

I have data on two processes, where the process assigns elements into ordered bins. I am interested in testing for agreement between the processes. What is the best way to do this (R code)? Here is ...