1
$\begingroup$

I have a study where the goal is to see if a new diagnostic tool provides any addition to the old procedure. A set of patients have been diagnosed and classified into 7 categories, each corresponding to a given treatment. The new diagnostic tool is then used, and the patients are diagnosed and classified again. Patients are then classified into two categories:

  1. the suggested treatment remains the same; or
  2. the suggested treatment is changed.

It seems as if this should be a simple question, but still I can't figure out what kind of test I could use to see if the diagnostic tool provides any additional information leading to revision of treatment. Given that there is a test, I would also like to do some sample size calculations for the next part of the study.

$\endgroup$
2
  • $\begingroup$ Is there any way of knowing which classification is correct (a so-called 'gold standard')? Are you wanting to know if the new tool is an improvement, or only if the 2 procedures agree? $\endgroup$ Jan 2, 2013 at 13:20
  • $\begingroup$ Only agreement is considered, it is not possible to tell which procedure is the better. $\endgroup$ Jan 2, 2013 at 15:57

1 Answer 1

1
$\begingroup$

You can think of the two different diagnostic tools as different 'raters', and assess the interrater agreement. The prototypical analysis is Cohen's kappa. Jon Uebersax has a webpage dedicated to interrater agreement with lots of good information; you'll probably want to spend some time there learning more about it.

If you use R, there is the irr package for assessing interrater agreement. It has functions for calculating Cohen's kappa (?kappa2), and for calculating power / sample size (?N2.cohens.kappa).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.