1
$\begingroup$

I have data set of teacher's evaluation rated by 4 different raters. The teacher's are evaluated for 13 different categories for example (Interaction with students, lesson delivery etc). All of the 4 raters are rating teachers in 13 categories from 1 to 5.

I want to find the agreement level between observers using cohen's kappa. I know how to compute kappa for one category only but I am confused how can we do that for different categories? Do I have to compute it for each category separately? or is there any other method?

For computing kappa I am using STATA.Here's an example Here's an example of how my data looks like. Can't share the original data.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

From kappa - Stata "kap (second syntax) and kappa calculate the kappa-statistic measure when there are two or more (nonunique) raters and two outcomes, more than two outcomes when the number of raters is fixed, and more than two outcomes when the number of raters varies. kap (second syntax) and kappa produce the same results; they merely differ in how they expect the data to be organized."

So, yes you can. Also, this other answer may help you with interpretation.

Improve this answer
$\endgroup$
3
  • $\begingroup$ So I have to calculate it separately for each category? $\endgroup$
    – Asra Khalid
    Mar 25, 2020 at 5:14
  • 1
    $\begingroup$ No. See for example statisticssolutions.com/KappaCalculator.html $\endgroup$
    – Carl
    Mar 25, 2020 at 5:56
  • $\begingroup$ Maybe read through the SPSS documentation on it beginning here, and look at what they do as well by following the links to using SPSS. It isn't STATA, but SPSS documentation is generally good. $\endgroup$
    – Carl
    Mar 25, 2020 at 6:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.