1
$\begingroup$

Coming from Stackoverflow-

So, I am creating a SVM model for a highly imbalanced data set and trying to create to calculate F, Pression and recall from the confusion matrix of the model. Confusion matrix is shown below and graph generated for this model is also attached. My question is, the calculated F, Precision and recall from the Confusion Matrix is very high (more than 97%) but the graph says less.

Why is that ? Any inputs, am I missing something here.

 Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 1104    2
         1    0    6

enter image description here

$\endgroup$
0
$\begingroup$

I think there is some misinterpretation happening regarding both the graph and how the Precision, Recall F1-score variables from the presented confusion matrix are calculated.

Starting with the confusion matrix, the Precision is 100% but actually the Recall is at 75% (6/(6+2)). So the F1 score is actually 85.7% still "great" generally speaking but not "more than 97%". Focusing now on the P-R curve shown: we can see it conveys similar information. From the confusion matrix we can see that we have 8 Positive examples. Effectively up until reaching 62.5% (5/8) we have 100% Precision, i.e. we detect the first 4 Positive examples fine using a very high threshold but we have lower our threshold to something close to 0.33 (judging from the greenish colour) to detect our next Positive example and increase our Recall further. That nevertheless drops our Precision significantly because we accumulate our False Positive somewhere around there too. As we further lower our threshold to values close to 0, we get some bouncy behaviour as our Precision is artificially inflated at times but realistically we get lower and lower Precision up until the point we reach 100% Recall.

As a conclusion, an average precision score of 0.81.9% seems ball-park correct for this classifier. The confusion matrix (most likely calculated a threshold of 0.5) also concurs to that with a F1 score of about 85.7%.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.