Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have data on two processes, where the process assigns elements into ordered bins. I am interested in testing for agreement between the processes. What is the best way to do this (R code)? Here is the dummy data with counts for the 2 processes and 4 categories). I am not familiar with weighted kappa (which I have seen argued by some as the test to use) - so if that is correct, can anyone explain the way to use this method?

enter image description here

EDIT: Adding to this question, I have the following R code. Is this proper and is this value suggestive of agreement?

table<-matrix(c(35,2,10,7,6,15,8,6,4,5,12,8,1,0,0,5),4,4, byrow=TRUE)

With result:

               value        ASE
Unweighted 0.3630137 0.06201821
Weighted   0.3854999 0.09886236
share|improve this question

Weighted kappa is one possible solution, another is to use gamma coefficient by Goodman and Kruskal that is applied to pair of ordered variables.

See also vcd and vcdExtra R packages for GKgamma() function that directly computes Goodman Kruskal gamma. The function also returns significance, what you need in your application. There is also possiblility for nice visualization of agreement between two variables, see manual for details.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.