# NaN produced from Cohen Kappa without 100% agreement

Considering the following data frame called coris_d0:

structure(list(RDT = c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1),
ELISA = c(1, 1, 1, 0, NA, 1, NA, 1, 1, 1, 1, 1, 1, 0, 1,
0, 1, 1, 1, 1, 0, NA, 1, 1, 1, 0, NA, 1, 1, 0, 1, NA, 1)), .Names = c("RDT",
"ELISA"), row.names = c(NA, -33L), class = "data.frame")


I want to check the agreement between the two methods (RDT and ELISA), so I am carrying out a Cohen Kappa from the psych package. If I run the following commands:

library("psych")
kappa2(coris_d0)


I wil get the following:

 Cohen's Kappa for 2 Raters (Weights: unweighted)

Subjects = 28
Raters = 2
Kappa = 0

z = NaN
p-value = NaN


This question here is quite similar and I agree with the explanation. However, having a quick look at my data set we can see that there is no 100% agreement, so I do not understand why I am getting this "error message".

I don't think the "NAs" are upsetting the code because if I run the same command on the following data frame (which also contains NAs):

 structure(list(RDT = c(1, 0, 1, 0, 1, NA, 1, 1), ELISA = c(1,
0, 1, 1, 1, 1, 1, 1)), .Names = c("RDT", "ELISA"), row.names = c(NA,
-8L), class = "data.frame")


I will get:

 Cohen's Kappa for 2 Raters (Weights: unweighted)

Subjects = 7
Raters = 2
Kappa = 0.588

z = 1.71
p-value = 0.0877


Moreover, we can see that the NA was removed from the analysis.

• Just a quick remark kappa2 is a function of the irr package, not psych (which has a function called cohen.kappa). I had the same problem as you and tested different packages as well :) – jkd Jan 5 at 17:57

This is because my two "raters" should have at least two levels, which was not an error message from the kappa2 function. If I run the following command:

library(fmsb)
Kappa.test(x = rep(1,33), y = c(rep(1,3),0,NA,1,NA,rep(1,6),0,1,0,1,1,1,1,0,NA,rep(1,3),0,NA,1,1,0,1,NA,1))


I will have the following error message:

Error in Kappa.test(x = rep(1, 33), y = c(rep(1, 3), 0, NA, 1, NA, rep(1,  :
'x' and 'y' must have at least 2 levels


Because one of my raters (x) has only one level ("1").