Proof of lower bound of Cohen's kappa

Question

Can anyone refer me to a paper or book showing that Cohen's kappa is bounded below by $-1$? I've read various papers stating this, but I have never found a complete proof. This question was asked here, but the answer only shows that $\kappa \leq 1$, which is very straightforward. Any help would be much appreciated!

In his paper introducing $\kappa$, Cohen states that "...its lower limit falls between zero and $-1.00$," but he only shows this lower limit when $r_m$ (the product-moment correlation of the two raters' marginal probabilities, regarded as paired observations $(p_{i,A}, p_{i,B})_{i=1}^k$) is 0 or negative, not when it is positive. For those able to read the paper, note that Cohen does not show $\kappa_{1+} \geq -1$ in general.

I believe I've managed to show $\kappa \geq -1$, but the algebra is somewhat painful. I may share my work here if no reference is forthcoming. (It would be redundant otherwise!)

(If you're wondering why I'm concerned with negative values of $\kappa$, I want to write code to verify some power calculations for $\kappa$, and equation solvers such as uniroot() in R ask for an interval in which to seek solutions.)

Answer 1

I still haven't found a reference. Here is my proof:

Consider a rating with $K$ categories. Let $n_{i,j} \geq 0$ be the number of observations assigned to category $i$ by rater 1 and category $j$ by rater 2. Let $n = \sum_{i,j} n_{i,j}$ be the total number of observations. Then $\kappa \geq -1$ is equivalent to $p_o - 2 p_e + 1 \geq 0$. Multiplying both sides of the inequality by $n^2$ yields

$$ \bigg( \sum_{i,j} n_{i,j} \bigg) \bigg( \sum_k n_{k,k} \bigg) - 2 \sum_i \bigg( \sum_j n_{i,j} \bigg) \bigg( \sum_k n_{k,i} \bigg) + \bigg( \sum_{i,j} n_{i,j} \bigg) \bigg( \sum_{k,\ell} n_{k,\ell} \bigg) \geq 0 , $$

with all summations running from $1$ through $K$. The left-hand side, say $S$, is a polynomial in the counts $n_{i,j}$ with second-order terms only. Define

$$ \begin{align*} A_1 &= \bigg( \sum_{i,j} n_{i,j} \bigg) \bigg( \sum_k n_{k,k} \bigg) , \\ A_2 &= \sum_i \bigg( \sum_j n_{i,j} \bigg) \bigg( \sum_k n_{k,i} \bigg) , \\ A_3 &= \bigg( \sum_{i,j} n_{i,j} \bigg) \bigg( \sum_{k,\ell} n_{k,\ell} \bigg) , \end{align*} $$

The only terms in $S$ that may have negative coefficients are those appearing in $A_2$. All such terms take one of the following forms, for distinct $a, b, c \in \{1, \dotsc, K\}$:

1. $n_{a,a}^2$. This term has coefficient 0 in $S$, since it appears once in each of the summations $A_1$, $A_2$, and $A_3$.
2. $n_{a,a} n_{a,b}$ or $n_{a,a} n_{b,a}$. A term of this form has coefficient 1, since it appears once in $A_1$, once in $A_2$, and twice in $A_3$.
3. $n_{a,b} n_{b,a}$. This term has coefficient -2, since it appears zero times in $A_1$, twice in $A_2$, and twice in $A_3$.
4. $n_{a,b} n_{c,a}$. This term has coefficient 0, since it appears zero times in $A_1$, once in $A_2$, and twice in $A_3$.

Finally, observe that the coefficient of $n_{a,b}^2$, where $a \neq b$, is 1, since it appears once in $A_3$ only. Keeping only terms of this form and of form (3), and dropping all other terms (which have nonnegative coefficients, hence are nonnegative), we have

$$ S \geq \sum_{i \neq j} n_{i,j}^2 - 2 \sum_{i < j} n_{i,j} n_{j,i} = \sum_{i < j} (n_{i,j} - n_{j,i})^2 \geq 0 , $$

equivalent to $\kappa \geq -1$.