# Inter-rater reliability simultaneously on multiple binary data

I have drug prescription records on 5 different drugs (A,B,C,D,E) on 2 different time points over a bunch of patients. 1/0 stands for use or not-use the specific drug, respectively.

My questions are, If I want to estimate the inter-rater reliability of 2 different time points simultaneously on 5 different drugs at same time, what test should I apply?

Should I separately calculate the kappa on each drug and reported them individually ? or Could I just average the kappa score (geometric or arithmetic mean?)??

Thank you ~

• Are the drugs mutually-exclusive or could a patient have more than one at a time? Jun 17, 2016 at 1:34
• I'm not quit understanding the meaning of mutually-exclusive here.... It is possible for each patient to receive more than one drug at a time. However, patient would have only 1 combination of drugs (A,B,C,D,E) at one time: e.g. patient would be prescribed drug A,D and E at time 1. Coding as (A,B,C,D,E)=(1,0,0,1,1) . Jun 17, 2016 at 12:32