For my problem I have a group of
p participants and
s subjects. I was suggested to calculate the Kappa Cohen Test on combinations of two for
s, then calculate the
mean and compare against the uniquely calculated value of Fleiss` kappa for the same set.
My question is, whetheror not it is correct/safe to calculate a mean over the Cohen's kappa test?
More specifically, I am using the Kappa2 and Fleiss' kappa with the flag set to FALSE from the irr R package. Setting the flag to false means that a variant algorithm of Kappa Fleiss is used so that it can be reduced to Kappa Cohen when the population has p=2 s=2.