How to compare the effectiveness of medical diagnostic techniques? A pathologist friend came to me for help with the following question for a research project.  The goal is to compare the effectiveness of three different diagnostic techniques.  The data set is as follows: there are 50 different specimens, each specimen was evaluated by 4 pathologists, and 3 different instruments (ie 600 total diagnoses).  Each case has a possible diagnosis of positive or negative, and the true results are known, as they have been independently determined.  The success rate depends on both the quality of the instrument and the skill of the pathologist, and we cannot assume that the four pathologists have the same proficiency.  Finally, even though each person measured the same specimen 3 times, they can be treated as independent measurements.
What are the appropriate tests for comparing effectiveness among the instruments?
Thanks.
ADDED:
Lots of good info in the answers, thanks to both.  Any thoughts on how ROC and randomized block compare/contrast? 
I'm not sure I've digested it enough to know which method is "better". Since the results need to be communicated to a certain audience, it probably depends on which is more widely used among that audience.
 A: As it is described in the original post, the experiment is a randomized block.


*

*Pathologist (4 levels) is a blocking factor; the experiment is repeated within each pathologist.

*Instrument (3 levels) and the true result (2 levels) of the test are the two treatments, which I assume were assigned randomly.

*Consider the different specimens to be replications of each treatment combination.

*The one response variable is whether the pathologist's diagnosis is correct (2 levels).


Because the result is categorical, the link function will need to be something like logit or probit. Here's some R code that does that. It may need to be extended depending on your friend's hypotheses.
library(lme4)
glmer(correctness ~ instrument*trueresult + (1|pathologist),
  family = binomial)

The coefficients from a logit model can be interpreted in relation to odds ratios. For a particular combination of predictors, the model estimates an odds ratio. The individual coefficients indicate how the odds ratio changes depending on the predictors.
If your friend doesn't care about distinguishing between type I and type II error, he or she can drop true result predictor from the model.
library(lme4)
glmer(correctness ~ instrument+trueresult + (1|pathologist),
  family = binomial)

The measurement in multiple sessions may be an incomplete block design, so your friend should look at those if he or she is concerned about the assumption of independence among measurements.
A: The ROC curve (Receiver Operating Characteristics) is one of the techniques available. You can check the questions with the tag roc on this site for further details. The wikipedia article
 http://en.wikipedia.org/wiki/Receiver_operating_characteristic and the external links to it may also be useful.
Some other methods can be found here 
http://onbiostatistics.blogspot.com/2011/01/agreement-statistics-and-kappa.html
