0
$\begingroup$

I have drug prescription records on 5 different drugs (A,B,C,D,E) on 2 different time points over a bunch of patients. 1/0 stands for use or not-use the specific drug, respectively.

My questions are, If I want to estimate the inter-rater reliability of 2 different time points simultaneously on 5 different drugs at same time, what test should I apply?

Should I separately calculate the kappa on each drug and reported them individually ? or Could I just average the kappa score (geometric or arithmetic mean?)??

Thank you ~

Improve this question
$\endgroup$
2
  • $\begingroup$ Are the drugs mutually-exclusive or could a patient have more than one at a time? $\endgroup$
    – Jeffrey Girard
    Jun 17, 2016 at 1:34
  • $\begingroup$ I'm not quit understanding the meaning of mutually-exclusive here.... It is possible for each patient to receive more than one drug at a time. However, patient would have only 1 combination of drugs (A,B,C,D,E) at one time: e.g. patient would be prescribed drug A,D and E at time 1. Coding as (A,B,C,D,E)=(1,0,0,1,1) . $\endgroup$
    – Hsuan Su
    Jun 17, 2016 at 12:32

1 Answer 1

Reset to default
1
$\begingroup$

Because your categories are not mutually-exclusive (i.e., it is possible for each patient to receive more than one drug at a time), it would be most appropriate to calculate agreement (or kappa, pi, etc.) for each drug separately. Another option. albeit a less commonly used and more complex one, would be to use the approach of Kraemer (1980) setting all drugs as tied for first. See my answer here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.