1
$\begingroup$

I need to ask about how many categories can be included in a weighted kappa or cohens kappa statistic for inter observer reliability.

Effectively we need to be reliable in assessing whether the distance between two monkeys is: 0m, 1m, 3m, 5m or >5m.

Normally for a cohens kappa it is a binomial agree/disagree although I have seen a cohens kappa done with three categories. I obviously have 5 categories and I wondered if it was possible to calculate a weighted cohens kappa result for my data which has the 5 categories above?

If so how could this be calculted either in excel or R?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Yes, there is no effective limit on the number of categories for Cohen's kappa but if you have many categories which are ordered (as you have in your example) then it might be better to use the intra class correlation. Note also that you may have problems if one of your raters does not use one of the categories.

This site does not answer programming questions but you can do this in R. As someone once wrote - This is R - the question is not Can? but How?. Try searching for R package Cohen kappa. I usually use the psych package available from CRAN.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.