There is a lot of misunderstanding about evaluation. Part of this comes from the Machine Learning approach of trying to optimize algorithms on datasets, with no real interest in the data.
In a medical context, it's about the real world outcomes - how many people you save from dying, for example. In a medical context Sensitivity (TPR) is used to see how many of the positive cases are correctly picked up (minimizing the proportion missed as false negatives = FNR) while Specificity (TNR) is used to see how many of the negative cases are correctly eliminated (minimizing the proportion found as false positives = FPR). Some diseases have a prevalence of one in a million. Thus if you always predict negative you have an Accuracy of 0.999999 - this is achieved by the simple ZeroR learner that simply predicts the maximum class. If we consider the Recall and Precision for predicting that you are disease free, then we have Recall=1 and Precision=0.999999 for ZeroR. Of course, if you reverse +ve and -ve and try to predict that a person has the disease with ZeroR you get Recall=0 and Precision=undef (as you didn't even make a positive prediction, but often people define Precision as 0 in this case). Note that Recall (+ve Recall) and Inverse Recall (-ve Recall), and the related TPR,FPR,TNR & FNR are always defined because we are only tackling the problem because we know there are two classes to distinguish and we deliberately provide examples of each.
Note the huge difference between missing cancer in the medical context (someone dies and you get sued) versus missing a paper in a web search (good chance one of the others will reference it if its important). In both cases these errors are characterized as false negatives, versus a large population of negatives. In the websearch case we will automatically get a large population of true negatives simply because we only show a small number of results (e.g. 10 or 100) and not being shown shouldn't really be taking as a negative prediction (it might have been 101), whereas in the cancer test case we have a result for every person and unlike websearch we actively control the false negative level (rate).
So ROC is exploring the tradeoff between true positives (versus false negatives as a proportion of the real positives) and false positives (versus true negatives as a proportion of the real negatives). It is equivalent to comparing Sensitivity (+ve Recall) and Specificity (-ve Recall). There is also a PN graph which looks the same where we plot TP vs FP rather than TPR vs FPR - but since we make the plot square the only difference is the numbers we put on the scales. They are related by constants TPR=TP/RP, FPR=TP/RN where RP=TP+FN and RN=FN+FP are the number of Real Positives and Real Negatives in the dataset and conversely biases PP=TP+FP and PN=TN+FN are the number of times we Predict Positive or Predict Negative. Note that we call rp=RP/N and rn=RN/N the prevalence of positive resp. negative and pp=PP/N and rp=RP/N the bias to positive resp. negative.
If we sum or average Sensitivity and Specificity or look at the Area Under the tradeoff Curve (equivalent to ROC just reversing the x-axis) we get the same result if we interchange which class is +ve and +ve. This is NOT true for Precision and Recall (as illustrated above with disease prediction by ZeroR). This arbitrariness is a major deficiency of Precision, Recall and their averages (whether arithmetic, geometric or harmonic) and tradeoff graphs.
The PR, PN, ROC, LIFT and other charts are plotted as parameters of the system are changed. This classically plot points for each individual system trained, often with a threshold being increased or decreased to change the point at which an instance is classed positive versus negative.
Sometimes the plotted points may be averages over (changing parameters/thresholds/algorithms of) sets of systems trained in the same way (but using different random numbers or samplings or orderings). These are theoretical constructs that tell us about the average behaviour of the systems rather than their performance on a particular problem. The tradeoff charts are intended to help us choose the correct operating point for a particular application (dataset and approach) and this is where ROC gets its name from (Receiver Operating Characteristics aims to maximize the information received, in the sense of informedness).
Let us consider what Recall or TPR or TP can be plotted against.
TP vs FP (PN) - looks exactly like the ROC plot, just with different numbers
TPR vs FPR (ROC) - TPR against FPR with AUC is unchanged if +/- are reversed.
TPR vs TNR (alt ROC) - mirror image of ROC as TNR=1-FPR (TN+FP=RN)
TP vs PP (LIFT) - X incs for positive and negative examples (nonlinear stretch)
TPR vs pp (alt LIFT) - looks the same as LIFT, just with different numbers
TP vs 1/PP - very similar to LIFT (but inverted with nonlinear stretch)
TPR vs 1/PP - looks the same as TP vs 1/PP (different numbers on y-axis)
TP vs TP/PP - similar but with expansion of the x-axis (TP = X -> TP = X*TP)
TPR vs TP/PP - looks the same but with different numbers on the axes
The last is Recall vs Precision!
Note for these graphs any curves that dominate other curves (are better or at least as high at all points) will still dominate after these transformations. Since domination means "at least as high" at every point, the higher curve also has "at least as high" an Area under the Curve (AUC) as it includes also the area between the curves. The reverse is not true: if curves intersect, as opposed to touch, there is no dominance, but one AUC can still be bigger than the other.
All the transformations do is reflect and/or zoom in different (non-linear) ways to a particular part of the ROC or PN graph. However, only ROC has the nice interpretation of Area under the Curve (probability that a positive is ranked higher than a negative - Mann-Whitney U statistic) and Distance above the Curve (probability that an informed decision is made rather than guessing - Youden J statistic as the dichotomous form of Informedness).
Generally, there is no need to use the PR tradeoff curve and you can simply zoom into the ROC curve if detail is required. The ROC curve has the unique property that the diagonal (TPR=FPR) represents chance, that the Distance above the Chance line (DAC) represents Informedness or the probability of an informed decision, and the Area under the Curve (AUC) represents Rankedness or the probability of correct pairwise ranking. These results do not hold for the PR curve, and the AUC gets distorted for higher Recall or TPR as explained above. PR AUC being bigger does not imply ROC AUC is bigger and thus does not imply increased Rankedness (probability of ranked +/- pairs being correctly predicted - viz. how often it predicts +ves above -ves) and does not imply increased Informedness (probability of an informed prediction rather than a random guess - viz. how often it knows what it's doing when it makes a prediction).
Sorry - no graphs! If anyone wants to add graphs to illustrate the above transformations, that would be great! I do have quite a few in my papers about ROC, LIFT, BIRD, Kappa, F-measure, Informedness, etc. but they aren't presented in quite this way although there are illustrations of ROC vs LIFT vs BIRD vs RP in https://arxiv.org/pdf/1505.00401.pdf
UPDATE: To avoid trying to give full explanations in overlong answers or comments, here are some of my papers "discovering" the problem with Precision vs Recall tradeoffs inc. F1, deriving Informedness and then "exploring" the relationships with ROC, Kappa, Significance, DeltaP, AUC, etc. This is a problem one of my students bumped into 20 years ago (Entwisle) and many more have since found that realworld example of their own where there was empirical proof that the R/P/F/A approach sent the learner the WRONG way, while Informedness (or Kappa or Correlation in appropriate cases) sent them the RIGHT way - now across dozens of fields. There are also many good and relevant papers by other authors on Kappa and ROC, but when you use Kappas versus ROC AUC versus ROC Height (Informedness or Youden's J) is clarified in the 2012 papers I list (many of the important papers of others are cited in them). The 2003 Bookmaker paper derives for the first time a formula for Informedness for the multiclass case. The 2013 paper derives a multiclass version of Adaboost adapted to optimize Informedness (with links to the modified Weka that hosts and runs it).
1998 The present use of statistics in the evaluation of NLP parsers.
J Entwisle, DMW Powers - Proceedings of the Joint Conferences on New Methods in Language Processing: 215-224
Cited by 15
2003 Recall & Precision versus The Bookmaker.
DMW Powers - International Conference on Cognitive Science: 529-534
Cited by 46
2011 Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation.
DMW Powers - Journal of Machine Learning Technology 2(1):37-63.
Cited by 1749
2012 The problem with kappa.
DMW Powers - Proceedings of the 13th Conference of the European ACL: 345-355
Cited by 63
2012 ROC-ConCert: ROC-Based Measurement of Consistency and Certainty.
DMW Powers - Spring Congress on Engineering and Technology (S-CET) 2:238-241
Cited by 5
2013 ADABOOK & MULTIBOOK: : Adaptive Boosting with Chance Correction.
DMW Powers- ICINCO International Conference on Informatics in Control, Automation and Robotics
https://www.dropbox.com/s/artzz1l3vozb6c4/weka.jar (goes into Java Class Path)
https://www.dropbox.com/s/dqws9ixew3egraj/wekagui (GUI start script for Unix)
https://www.dropbox.com/s/4j3fwx997kq2xcq/wekagui.bat (GUI shortcut on Windows)
Cited by 4