I have two databases of patients and medications. Each database has the same patients. Both have millions of records.

My job is to select a sample from each and confirm that the medications match. So medication is a nominal variable with a huge number of levels. Any advice on selecting a sample size for this and also statistics to use beyond percent matched/unmatched. Cohen's Kappa?

Does the number of levels of the nominal variable variable effect Kappa? Would conceiving of each medication as binary make it easier?

  • $\begingroup$ What are the consequences of the two types of errors? Compared to those consequences, what does it cost to sample the gold standard? What is the overall objective? $\endgroup$ – whuber Jan 12 at 21:02
  • $\begingroup$ Overall objective is to judge the reliability and validity of a dataset to see if it could be used for research and other purposes. There is no danger to humans, so no grave danger associated with either type of error. It is expensive in terms of time and money to sample to the gold standard. $\endgroup$ – Nw2this Jan 13 at 0:31
  • $\begingroup$ Unfortunately, "to see if it could be used for research and other purposes" doesn't tell us anything of use in answering this question. What quantitative criteria would apply to determining whether the database is suitable for those purposes? $\endgroup$ – whuber Jan 13 at 12:22
  • $\begingroup$ Basically, I just need a formula or reference for a sample size calculator for Kappa when there are more than 2 levels of the categorical variable. $\endgroup$ – Nw2this Jan 14 at 2:05
  • $\begingroup$ I discovered PASS software. This solved my problem. $\endgroup$ – Nw2this Jan 16 at 16:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.