# How to compare two Cohen's Kappa statistics based on the same sample of subjects but different raters

Suppose I have two assessment methods A and B for assessing depression. I first ask two raters (Raters 1 and 2) to use Method A to assess 50 patients. I compute Cohen's Kappa based on the ratings (assuming the ratings are ordinal). I then ask two different raters (Raters 3 and 4) to use Method B to assess the same 50 patients (again, let's assume the ratings are ordinal). I compute Cohen's Kappa based on these two new raters' ratings.

I wonder if there is a way to compare these two kappa statistics -- that is, whether they are statistically different.

• The link is broken, here are some possible R packages and functions: rel::ckap, irr::kappa2, or psych::cohen.kappa. Feb 4 at 3:37