Sample size calculation: interpreting the results for comparing AUCs I have N number of patients each who could have 1 of 5 diseases (A, B, C, D, E). There is clinical information that may improve the accuracy of doctor diagnoses of these N patients. All diagnoses will be confirmed. 
I want to test whether that information improves diagnostic accuracy.
To do this, I am going to calculate the AUC for each diagnosis for a doctor before he has this clinical information and after. I will calculate the percent agreements for each set of diagnoses. I want to be able to show that the AUC for each disease increases 15%. Based on previous studies, lets say I know the doctors' concordance statistic, or $c$-statistic, before given the clinical information is 70 units and the SD is 2 units. I want a power of 0.9 at a significance level of 0.05. 
I need to calculate the minimum sample size needed for this. In STATA, I have found that my N must be 46, randomized 1:1 to receive diagnoses without the new information and with the new information. My problem is exactly how to interpret this. This is really for the binary diagnosis of A versus not A. Therefore, for 5 different possibilities I will need 115 (5 x 23) of each diagnosis in each group. 
 A: I think in the last paragraph you stumbled onto the confusion I had from the outset. The AUC is not really a useful measure unless there is a continuous diagnostic score, like a PSA or a Framingham Risk Score, to evaluate over a range of possible diagnostic thresholds. 
Since this is a study where diagnosis is measured in terms of a "yes/no" outcome, you are actually better poised to conduct a more practical study. Risk scores, and their AUCs, have no clinical relevance. They must be translated into an action: diseased/healthy, treat/don't treat, insulin/diet, mastectomy/lumpectomy and chemotherapy. If the outcome is already on the "yes/no" scale you are done. We use different metrics to evaluate diagnostic performance.
The Cohen's Kappa is a measure of interrater reliability. When the comparison group is a diagnostic confirmation, what we also call a gold standard, then it is also a measure of recall. In fact, if the disease statuses, A, B, C, D, E, are ordinal categories, the Cohen's Kappa has multicategory analogues if, say, a diagnosis of C is closer to a disease status D than a diagnosis of A. This is, for example, useful for cancer staging studies where Stage III and Stage IV cancers usually receive aggressive treatment.
There are tools and methods on sample size calculations for these measures elsewhere on the internet:
https://www.ime.usp.br/~abe/lista/pdfGSoh9GPIQN.pdf
https://rdrr.io/cran/irr/man/N.cohen.kappa.html
