ROC vs precision-and-recall curves I understand the formal differences between them, what I want to know is when it is more relevant to use one vs. the other. 


*

*Do they always provide complementary insight about the performance of a given classification/detection system? 

*When is it reasonable to provide them both, say, in a paper? instead of just one?

*Are there any alternative (maybe more modern) descriptors that capture the relevant aspects of both ROC and precision recall for a classification system?


I am interested in arguments for both binary and multi-class (e.g. as one-vs-all) cases.
 A: Here are the conclusions from a paper by Davis & Goadrich explaining the relationship between ROC and PR space. They answer the first two questions:

First, for any dataset, the ROC curve and PR curve for a given algorithm contain the same points. This equivalence, leads to the surprising theorem that a curve dominates in ROC space if and only if it dominates in PR space. Second, as a corollary to the theorem we show the existence of the PR space analog to the convex hull in ROC space, which we call achievable PR curve. Remarkably, when constructing the achievable PR curve one discards exactly the same points omit- ted by the convex hull in ROC space. Consequently, we can efficiently compute the achievable PR curve. [...] Finally, we show that an algorithm that optimizes the area under the ROC curve is not guaranteed to optimize the area under the PR curve.

In other words, in principle, ROC and PR are equally suited to compare results. But for the example case of a result of 20 hits and 1980 misses they show that the differences can be rather drastic, as shown in Figures 11 and 12.

Result/curve (I) describes a result where 10 of the 20 hits are in the top ten ranks and the remaining 10 hits are then evenly spread out over the first 1500 ranks. Resut (II) describes a result where the 20 hits are evenly spread over the first 500 (out of 2000) ranks. So in cases where a result "shape" like (I) is preferable, this preference is clearly distinguishable in PR-space, while the AUC ROC of the two results are nearly equal.
A: The key difference is that ROC curves will be the same no matter what the baseline probability is, but PR curves may be more useful in practice for needle-in-haystack type problems or problems where the "positive" class is more interesting than the negative class.
To show this, first let's start with a very nice way to define precision, recall and specificity.  Assume you have a "positive" class called 1 and a "negative" class called 0.  $\hat{Y}$ is your estimate of the true class label $Y$.  Then:
$$
\begin{aligned}
&\text{Precision} &= P(Y = 1 | \hat{Y} = 1)  \\
&\text{Recall} = \text{Sensitivity} &= P(\hat{Y} = 1 | Y = 1)  \\
&\text{Specificity} &= P(\hat{Y} = 0 | Y = 0)
\end{aligned}
$$
The key thing to note is that sensitivity/recall and specificity, which make up the ROC curve, are probabilities conditioned on the true class label.  Therefore, they will be the same regardless of what $P(Y = 1)$ is.  Precision is a probability conditioned on your estimate of the class label and will thus vary if you try your classifier in different populations with different baseline $P(Y = 1)$.  However, it may be more useful in practice if you only care about one population with known background probability and the "positive" class is much more interesting than the "negative" class.  (IIRC precision is popular in the document retrieval field, where this is the case.)  This is because it directly answers the question, "What is the probability that this is a real hit given my classifier says it is?". 
Interestingly, by Bayes' theorem you can work out cases where specificity can be very high and precision very low simultaneously.  All you have to do is assume $P(Y = 1)$ is very close to zero.  In practice I've developed several classifiers with this performance characteristic when searching for needles in DNA sequence haystacks.
IMHO when writing a paper you should provide whichever curve answers the question you want answered (or whichever one is more favorable to your method, if you're cynical).  If your question is: "How meaningful is a positive result from my classifier given the baseline probabilities of my problem?", use a PR curve.  If your question is, "How well can this classifier be expected to perform in general, at a variety of different baseline probabilities?", go with a ROC curve.
A: TL;DR
$AUC_{PvR}$ highlights the amount of False Positives relative to the class size, whereas $AUC_{ROC}$ better reflects the total amount of False Positives independent of in which class they come up.
Definition
The $AUC_{ROC}$ (receiver operator) is the area under the curve of true positive to false positive rate.
The $AUC_{PvR}$ (precision vs recall) is the area under the curve of the precison to recall metrics.
As true-positive rate equals recall, they only differ in comparing the recall to either precision or false-positive rate
$$
\begin{align}AUC_{PvR} =& AUC\left(\frac{\mathbf{TP}}{\mathbf{TP}+\mathbf{FP}}, \frac{TP}{TP + FN}\right)= AUC \left( \textbf{precision}, \text{recall}\right) \\AUC_{ROC} =& AUC\left(\frac{\mathbf{FP}}{\mathbf{FP} + \mathbf{TN}}, \frac{TP}{TP+FN}\right)= AUC \left(\textbf{FPR}, \text{recall} \right)\end{align}
$$
Illustration
Consider an unbalanced multiclass dataset with a million instances. We are given two classifiers and observe their predictions at a fixed threshold for one of the classes:
Classifier A: 100 instances predicted (positive), 90 of which correctly (with 100 true cases)
Classifier B: 2000 instances predicted (positive), 90 of which correctly (with 100 true cases)
The respective values for the $AUC_{ROC}$ at the specific threshold will be
Classifier A: 0.9 TPR, 0.00001 FPR
Classifier: B: 0.9 TPR, 0.00191 FPR (gain of 0.0019)
The respective values for the $AUC_{PvR}$ for the specific threshold will be
Classifier A: 0.9 recall, 0.9 precision
Classifier B: 0.9 recall, 0.045 precision (gain of 0.855)
Discussion
As you can see, by choosing classifier B over A, the gain in false positive rate is comparably low compared to the gains observed in precision.
This is because the false-positive rate is a ratio of the false positives to the vast amount of true negatives, whereas the precision is a ratio of the false positives to the rather small amount of true positives.
Therefore the $AUC_{ROC}$ is more globally in that it is irritated by false positives relative to how many you could have drawn from the whole dataset. The $AUC_{PvR}$, however, is more local, in that it is irritated by false positives relative to how many positives you have set for the class at hand.
Your choice of metric, therefore, depends on what really irritates you more.
If you are irritated by a minority class having many false-positives relative to its class size $AUC_{PvR}$ will highlight this more thoroughly. (also false-positives of a majority class will have less impact)
If you are however more irritated by the global amount of false positives, the $AUC_{ROC}$ would be more indicative of how well you are doing.
