Cohen's kappa is a measurement of the degree to which 2 rating systems (typically people making ratings) agree. It adjusts for the probability that the raters would agree by chance alone if they were completely independent.

The sampling distribution of kappa is known, so the measure can be used as a statistical test of agreement.

The standard calculation of kappa is: $$ \kappa=\frac{p(\text{agreement})-p(\text{chance agreement})}{1-p(\text{chance agreement})} $$ There are also weighted versions of kappa, and other related measures of inter-rater agreement.

history | show excerpt | excerpt history