5
$\begingroup$

I want to learn about Receiver-Operator-Characteristic curves, and metrics. I have read through online webpages with some basics, and I have used MATLAB built-ins to create ROC plots. It tells me about area sums. I have also looked at the JMP module for classification and for Logistic plot which make ROC plots.

What is the canon here? What is a great textbook with both deep fundamentals and worked out theory? MATLAB and R are preferred. Who made the field? When? Why?

I'm looking for good references. Thanks.

Improve this question
$\endgroup$
  • 4
    $\begingroup$ The first thing is to learn when ROC curves are relevant/helpful. I see them being used when it would be much better to use risk prediction. They are not useful for determining a cutoff when a cutoff doesn't exist or when you are seeking a cutoff on an independent variable. In general, ROC curves are at odds with individual decisionmaking. $\endgroup$ – Frank Harrell Apr 15 '14 at 22:06
  • $\begingroup$ @FrankHarrell I like your comment, what resources do you suggest? $\endgroup$ – Alessandro Jacopson Aug 31 '17 at 12:44
  • 1
    $\begingroup$ ROC curves, instead of assisting with understanding, get in the way of understanding. See the medical diagnosis chapter in BBR, link at fharrell.com/p/blog-page.html . There are two related articles on my blog fharrell.com . $\endgroup$ – Frank Harrell Aug 31 '17 at 13:00
  • $\begingroup$ @FrankHarrell, The resource behind the link has been moved. Would you mind mentioning the specific articles? $\endgroup$ – Narfanar Dec 7 '18 at 21:48
  • 1
    $\begingroup$ Sorry; it's now here. The master page for all my links is fharrell.com/links. $\endgroup$ – Frank Harrell Dec 8 '18 at 13:47
5
$\begingroup$

I would start from this practically canonical paper by Davis and Goadrich The relationship between Precision-Recall and ROC curves.

Through this, the ROC origins can be tracked to this book: Evaluation of Diagnostic Systems: Methods from Signal Detection Theory which is unfortunately hardly accessible.

Improve this answer
$\endgroup$
  • $\begingroup$ The paper asserts that P-R curves are less easily misinterpreted over-optimistically, but it says that description is outside the scope of the text. Would you mind augmenting your answer with more material on P-R curves? $\endgroup$ – EngrStudent Apr 15 '14 at 20:04
  • $\begingroup$ You didn't ask about P-R curves. $\endgroup$ – Aksakal Apr 15 '14 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.