2
$\begingroup$

I know the difference between Precision and Recall metrics in Machine Learning. One optimizes on False Positives and other on False Negative. In Statistics it is called as optimizing on Type I or Type II error.

However, I am a but confused on under what circumstances one can get complete opposite Precision and Recall? Like Precision =1 and Recall=0?.

Let me iterate: 

precision = true positives / (true positives + false positives)

recall = true positives / (true positives + false negatives)

And here is the Confusion Matrix

     predicted
            (+)   (-)
            ---------
       (+) | TP | FN |
actual      ---------
       (-) | FP | TN |
            ---------

Now, if Precision is =1 for a classifier for the positive (1) class, that means, there are no FP and all predicted labels are TP.

Then how can for the same positive class the Recall be 0? If there is already some TP being predicted, in fact as per Precision all predicted ones are TP only, then for Recall, we would have numerator non zero, then under what circumstances can one get Recall 0 then for the same classifier positive class?

To give some context, I ran a Logistic regression classifier for a binary classification problem. I had some 23K training data with 774 features. 770 features are binary or dummy variables.

And this is the distribution of my class labels: 

1    20429
0    12559

And here is the confusion matrix and accuracy values after a 5 fold grid search on some 25 combination of Hyper parameter values. 

The mean train scores are [ 0.66883049  0.54314532  0.67008959  0.63187226  0.63100366  0.53165968
  0.54131812  0.55507725  0.5578254   0.57663273  0.57247462  0.57230056
  0.54402055  0.5762753   0.50925733  0.45781882  0.39366017  0.39037968
  0.3919818   0.38878762  0.39784982  0.39506755  0.48238147  0.38932944
  0.39801223]

The mean validation scores are [ 0.66445801  0.54107661  0.66878871  0.63184791  0.6305487   0.5291239
  0.53899788  0.55324585  0.55822615  0.57784418  0.57269066  0.57312373
  0.54536399  0.57593868  0.50790351  0.45727773  0.39318349  0.38906933
  0.39214413  0.38924256  0.39794725  0.39461262  0.4827855   0.38811658
  0.39812048]

The score on held out data is: 0.6687887055562773
 Hyper-Parameters for Best Score : {'alpha': 0.0001, 'l1_ratio': 0.45}

The accuracy of sgd on test data is: 0.37526523188845107

Classification Metrics for sgd :
             precision    recall  f1-score   support

          0       0.38      1.00      0.55      3712
          1       1.00      0.00      0.00      6185

avg / total       0.77      0.38      0.21      9897
Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ A recall of 0 indicates that there were no positive cases in the input data, and precision is 1 if there are no false positives. See this answer. $\endgroup$
    – redress
    Commented Jun 5, 2017 at 2:54
  • $\begingroup$ I don't understand. There are positive cases in input data right? some 20K if you see the class label distribution. if You say positive predicted, then also it's there since Precision =1. I dont understand what you meant here $\endgroup$
    – Baktaawar
    Commented Jun 5, 2017 at 3:54
  • $\begingroup$ Check Section 4.1 of Ullah, S, et al. (2018). Nondestructive Inspection of Reinforced Concrete Utility Poles with ISOMAP and Random Forest. Sensors, 18(10), 3463. $\endgroup$
    – Saeed Ullah
    Commented Oct 17, 2018 at 2:27
  • 1
    $\begingroup$ You've envisioned the problem as forced-choice classification. For the particular application is there no interest in tendencies (probabilities)? $\endgroup$
    – Frank Harrell
    Commented Jan 10, 2022 at 13:25

1 Answer 1

Reset to default
1
$\begingroup$

Then how can for the same positive class the Recall be 0? If there is already some TP being predicted, in fact as per Precision all predicted ones are TP only, then for Recall, we would have numerator non zero, then under what circumstances can one get Recall 0 then for the same classifier positive class?

Rounding. If you have one true positive, zero false positives, and the metrics themselves are rounded to two decimal places (as shown), precision is 1, recall is zero and accuracy is roughly 1/3 as shown.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.