0
$\begingroup$

I want to assess ability to identify tumor types in humans vs. a computers. There are 7 tumor types. I am interested in calculating agreement between the human examiners but I am not sure the best way to go about it. I have only ever seen ICC used in continuous measures, but there is no order to tumor types. One option is to assess agreement between examiners using a binary (0=incorrectly identified, 1=correctly identified) but I do not know how to calculate that.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Maybe (adjusted) Rand Index? $\endgroup$
    – user2974951
    May 5, 2021 at 8:11

1 Answer 1

Reset to default
1
$\begingroup$

A better option is to calculate Fleiss’s kappa (or Cohen's kappa if there are only two examiners) using the categorical response data of what each examiner guesses is the correct tumor type.

https://www.datanovia.com/en/lessons/fleiss-kappa-in-r-for-multiple-categorical-variables/

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.