# Cohen's Kappa, why not simple ratio

I was wondering if there is any particular reason why Cohen's Kappa is defined as this particular ration $\frac{p - r}{1 - r}$, where $p$ is the agreement rate between two, say, classifiers, and $r$ is the rate of agreeing at random.

Why is Kappa not defined, for instance, simply as $\frac{p}{r}$? Is the present form only for the upper limit for Kappa to be $1$?

## 2 Answers

$$\kappa = \frac{p_o - p_c}{1 - p_c}$$

This formula does have a deeper meaning and also restricts the range between -1 and 1 (not 0 and 1). Kappa is meant to be the ratio of observed non-chance agreement to possible non-chance agreement.

The numerator ($p_o - p_c$) is the observed non-chance agreement because $p_o$ is the total observed agreement and we are removing the estimated chance agreement $p_c$.

The denominator ($1 - p_c$) is the possible non-chance agreement because $1$ is the maximum value that $p_o$ could take on and we are removing the estimated chance agreement $p_c$.

I think that the ratio of observed agreement to chance agreement ($p_o/p_c$) would be reasonable; it just wouldn't be bounded.

Cohen Kappa is defined so $\kappa$=0 means a random agreement, and $\kappa$=1 is the maximum possible Kappa, that is 100% accuracy.

As far as I know there is no deeper principle behind the formula - just the requirement described (0 -> random, 1 -> maximum). The wikipedia page onb Cohen's kappa list a few other inter-rater agreement measures and links to some papers that describe problems with the measure.

• Is it just for easy interpretation? – inzl Jan 12 '16 at 14:35
• I believe that is correct. The formula forces the value to be between 0 and 1. – William Chiu Jan 12 '16 at 16:19
• I would not put as " just for easy interpretation". Cohen wanted a number with the properties above and thus the formula - so the formula is that because Cohen wanted a number that could be interpreted as 0 -> random 1-> maximum. – Jacques Wainer Jan 12 '16 at 20:54