There is one thing which confuses me about two very common explanations regarding the interpretation of the Area Under The Receiver Operating Characteristic (referred to shortly as AUC). Concretely, these are
1) The benchmark of 0.5 for a coin toss, i.e. that if a simple coin toss is determined as a "test", that in this case the AUC-value would be 0.5.
2) That the AUC-value corresponds to the probability of correctly classifying two randomly individuals, one from each group (e.g. a person who has the disease a person who does not have the disease). For the matter of this post, the alternative explanation that, using normalized units, the AUC area under the curve is equal to the probability that a classifier will rank a randomly chosen positive instance higher than a randomly chosen negative one (assuming 'positive' ranks higher than 'negative').
What confuses me is the following thought: Suppose I use the coin-toss "test" and apply it two the two individuals from property 2.). This test has a probability of 0.5 for a false negative and 0.5 for a false positive. The probability that I classify a positive and a negative outcome correctly is thus 0.25 (Prob[no false negative for the positive indivudual] * Pr[no false positive for the negative individual]). In all other cases, the two scores/tests will be either equal or have results opposite to what is actually true (interpreting the test inversely as often suggested when the area is <0.5 does not help either for the coin flip case). This seems to be at odds with statement 1).
Of course, I could just pick one of the two individuals, test it, and assign it the outcome the test shows - which is correct in 50% of all cases - and assign the other individuals the opposite outcome. This would achieve an accuracy of 0.5 as described in 1) but it would not really be what is described in 2) since I am only testing one person (i.e., one person in the sample would not have a test-score).
The two paragraphs of reasoning seem to contradict each other. Is any of them wrong, if yes where or is this really a paradox?
*Note that I see that the coin toss would be under the diagonal and would get, together with other tests where the false-positive rate equals the true-positive rate, get an area of 0.5 under curve. It is just about the interpretation aides listed above and how to reconcile them.