Optimise SVM to avoid false-negative in binary classification

Question

I am training an SVM binary classifier using Scikit learn.

Due to the nature of my problem I need to avoid false negatives. As nothing is free I am okay getting a higher rate of false positives in order to reduce the number of false negatives. How can we do that (ideally with Scikit learn)?

In other words, how can we minimise false negatives using an SVM classifier ? Is there some way to tune hyper parameters in order to favor false positives over false negatives?

Answer 1

Scikit learn implementation of the SVM binary classifier does not let you set a cutoff threshold as the other comments/replies have suggested. Instead of giving class probabilities, it straighaway applies a default cutoff to give you the class membership e.g. 1 or 2.

To minimize false negatives, you could set higher weights for training samples labeled as the positive class, by default the weights are set to 1 for all classes. To change this, use the hyper-parameter class_weight .

Ideally, you should avoid choosing a cutoff and simply provide the class probabilities to the end users who can then decide on which cutoff to apply when making decisions based on the classifier.

A better metric to compare classifiers is a proper scoring function, see https://en.wikipedia.org/wiki/Scoring_rule and the score() method in the svm classifier module sklearn.svm.SVC.

Answer 2

Like many predictive model, SVM will output probability scores and the apply threshold to probability to convert it into positive or negative labels.

As, @Sycorax mentioned in comment, you can adjust the cut-off threshold to adjust the trade-off between false positive and false negative.

Here is some example in R.

library(kernlab)
library(mlbench)
graphics.off()
set.seed(0)

d=mlbench.2dnormals(500)
plot(d)
# using 2nd order polynominal expansion
svp <- ksvm(d$x,d$classes,type="C-svc",kernel="polydot",
            kpar=list(degree=2),C=10,prob.model=T)
plot(svp)

p=predict(svp,d$x, type="prob")[,1]

cut_off=0.5
caret::confusionMatrix(d$classes,ifelse(p<cut_off,2,1))

cut_off=0.8
caret::confusionMatrix(d$classes,ifelse(p<cut_off,2,1))

Note when we change cut_off, the confusion matrix (false postive, false negative etc.) changes

> caret::confusionMatrix(d$classes,ifelse(p<cut_off,2,1))
Confusion Matrix and Statistics

          Reference
Prediction   1   2
         1 253  16
         2  38 193

               Accuracy : 0.892           
                 95% CI : (0.8614, 0.9178)
    No Information Rate : 0.582           
    P-Value [Acc > NIR] : < 2.2e-16       

                  Kappa : 0.7813          
 Mcnemar's Test P-Value : 0.004267        

            Sensitivity : 0.8694          
            Specificity : 0.9234          
         Pos Pred Value : 0.9405          
         Neg Pred Value : 0.8355          
             Prevalence : 0.5820          
         Detection Rate : 0.5060          
   Detection Prevalence : 0.5380          
      Balanced Accuracy : 0.8964          

       'Positive' Class : 1               


> cut_off=0.8

> caret::confusionMatrix(d$classes,ifelse(p<cut_off,2,1))
Confusion Matrix and Statistics

          Reference
Prediction   1   2
         1 223  46
         2  10 221

               Accuracy : 0.888          
                 95% CI : (0.857, 0.9143)
    No Information Rate : 0.534          
    P-Value [Acc > NIR] : < 2.2e-16      

                  Kappa : 0.7772         
 Mcnemar's Test P-Value : 2.91e-06       

            Sensitivity : 0.9571         
            Specificity : 0.8277         
         Pos Pred Value : 0.8290         
         Neg Pred Value : 0.9567         
             Prevalence : 0.4660         
         Detection Rate : 0.4460         
   Detection Prevalence : 0.5380         
      Balanced Accuracy : 0.8924         

       'Positive' Class : 1