# How should I interpret Fleiss' kappa when it equals NaN?

I noticed that when I have tables which the values are only 0 and 1, I get a kappa of 1 when the table is completely full of one, and when I have a table of zeros I get NaN as result using the irr package and the kappa fleiss function.

I would expect the kappa to be also equals to 1 when the table is full of 0, as the 0 would represent agreement on 'No' and the full of 1s would represent agreement of 'Yes'.

What is the right way to interpret the results?

This is an issue with the formulation of many chance-adjusted indices of inter-rater reliability that estimate chance agreement using a distribution-based approach. Basically, you are getting a score of NaN because, with only one observed response category, $\kappa$ estimates the chance agreement $(p_c)$ as $1$. Using the formula below, a $p_c$ of $1$ results in a denominator of $0$, which cannot be computed:
$$\kappa = \frac{p_a - p_c}{1 - p_c}$$
In this case, there is perfect observed agreement (i.e., $p_a=1$) and I would report that instead of $\kappa$.