1
$\begingroup$

I am evaluating a classification model, Support Vector Machine, and I am having difficulty interpreting the Precision and Recall Curve 1 graph.

For example, this graph plots a straight negative line. I wasn't expecting this, especially when the accuracy of this model is 0.99, and it has a false negative rate of almost 0, and I also don't have false positives. That is, true positives are at 100% and true negatives are at 99.99%. You can check this info in the confusion matrix 2. How do I interpret a graph like this? Is my model completely wrong?

enter image description here enter image description here

$\endgroup$
1
  • $\begingroup$ Do you, by any chance, have a heavy class imbalance? $\endgroup$
    – Dave
    Mar 15 at 23:19

1 Answer 1

1
$\begingroup$

First notice that your y axis doesn't start at zero.

A generally negative slope is to be expected in precision-recall space. When you set the decision threshold below zero, you classify everything as positive, so your recall is 1.0 and your precision is equal to the prevalence of the positive class*; when you set the decision threshold close to one, you classify nearly nothing as positive, so your recall approaches 0.0 and your precision is hopefully close to 1.0.

*That suggests a mismatch between the plot and your expectations: the plot appears to be treating "Working" as the positive class (note that $3485/(3485+13)\approx0.9963$ matches the lower-right point of the plot), while you seem to be treating "Malfunctioning" as the positive class ("I don't have false positives").

Lastly, it is odd that the plot should be a straight line. Perhaps your SVM has lots of ties (or you're not passing in the confidence scores, just the hard predictions?)?

$\endgroup$
1
  • 1
    $\begingroup$ After reading your last line, I think that's the culprit: the PR curve is just the top-left and bottom-right points, and then the plotting software fills in between them with a straight line. $\endgroup$
    – Dave
    Mar 15 at 23:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.