# Rater's agreement for each item

I would like to calculate the agreement between 2 or more raters on all the judgments from a set, but also for each judgment.

For example:

rater1, rater2
aa, aa
bb, bb
cc, cd


I can use Cohen's Kappa or Fleiss Kappa to calculate the agreement for all the items, but can't find a way to calculate the agreement for each judgment. What R method should I use?

If I simply try to use Fleiss Kappa for the first row of the file ('aa', 'aa') I will get the following:

 Cohen's Kappa for 2 Raters (Weights: unweighted)

Subjects = 1
Raters = 2
Kappa = NaN

z = NaN
p-value = NaN


I would really need a function to do this agreement for each judgment.

You can calculate the percent observed agreement for a single item: $$A=\sum_{k=1}^q \frac{r_k(r_k-1)}{r(r-1)}$$ Where $q$ is the total number of categories, $r$ is the total number of raters that assigned the item to any category, and $r_k$ is the number of raters that assigned the item to category $k$.
Fleiss Kappa for 2 raters