Timeline for Inter-Rater Reliability/Agreement (100 Raters, 1 Item)
Current License: CC BY-SA 4.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 18 at 16:22 | comment | added | jginestet | That would not at all solve your main issue which is that you only have 1 "unit" being rated. And in addition, you would now sum ordinal numbers, which is, to say the least, a debatable practice. Note that you could also use Fleiss kappa (en.wikipedia.org/wiki/Fleiss%27_kappa), which can be computed even with only 1 subject (it computes agreement by chance differently). | |
| Oct 18 at 9:10 | comment | added | Marc Roux | Thank you very much for the explanations, which were very helpful. Would it therefore perhaps be wiser if I grouped the 5 criteria together? Category Delivery (= criterion eye contact and gestures) Category Conent (= criterion correctness, introduction etc.) In this way, I would not have the reliability for each criterion, but for individual areas and thus have more meaningful reliability data? | |
| Oct 17 at 17:57 | history | answered | jginestet | CC BY-SA 4.0 |