When to use a ROC Curve vs. a Precision Recall Curve?

Question

Looking for the circumstances of when we should use a ROC curve vs. a Precision Recall curve.

Example of answers I am looking for:

Use a ROC Curve when:

• you have a balanced or imbalanced dataset (Source).
• when the cost of false positives and false negatives is roughly equal (needs verification)
• ...

Use a Precision Recall Curve when:

• you have a imbalanced dataset with way more positives than negatives (Source).
• when the cost of false positives is higher than false negatives (needs verification)
• ...

Answer 1

Use both all the time.

There is no strong reason to pick one over another unless you have some kind of contractual obligation against one of the twoo. Both metrics have their pros and cons and those have been discussed at length in CV.SE (e.g. here, here and here) but neither of the two is a panacea for a particular situation. For example, why not use Brier Score or Continuous Ranked Probability Score (CRPS) too?

My advice is that if one really thinks that "generic metrics" like AUR-ROC, AUC-PR, Brier score, etc. are not fit for their modelling purposes then they have to consider cost-sensitive learning to account for significantly different misclassification costs and/or practical usefulness thus doing a proper decision curve analysis. Elkan (2001) The foundations of cost-sensitive learning is probably one of the most well-cited original papers on the matter. Practical usefulness is usually visited in the context of clinical applications so a good first read there is: Vickers & Elkin (2006) Decision Curve Analysis: A Novel Method for Evaluating Prediction Models.