what is the best performance mesure for spam filtering ?

Question

I have a SVM model for spam filtering and the resultat are like follows :

true negative : 115 , false positive : 15 false negative : 66, true positive : 64

We have these mesures:

True Positive Rate (TPR) or Hit Rate or Recall or Sensitivity = TP / (TP + FN)

False Positive Rate(FPR) or False Alarm Rate = 1 - Specificity = 1 - (TN / (TN + FP))

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Error Rate = 1 – accuracy or (FP + FN) / (TP + TN + FP + FN)

Precision = TP / (TP + FP)

F-measure = 2 / ( (1 / Precision) + (1 / Recall) )

Which one would be the most accurate if want to have the least important emails going to the spam folder and the least spams going to the important emails folder ?

I want to be less tolerant with having an important email going to spam folder

Answer 1

Your choice of performance measure is how you decide which of several models is the best, and more generally, how good the models are. To ask which of several possible performance measures is best or most accurate is to put the cart before the horse. Instead, think about how the various measures try to quantify good performance and which sort of performance you most want to see from your spam filter. The only way around this is to use a decision-theoretic or philosophical meta-criterion that tells you which criterion to choose—but then, of course, you have to choose a meta-criterion.