I am developing a fraud detection system for online purchases using random forests. My first concern was to optimize for recall, as the dataset was originally unbalanced (98% no fraud events and 2% fraud events). After resampling my dataset and forcing it to be "balanced" (40% fraud and 60% no fraud) it is now performing well in terms of recall (40%). The issue is that I now want to optimize for specificity. Why? Because the tradeoff of optimizing for recall is that I am declining a lot of no fraud transactions (i.e. sacrificing my acceptance rate).

I am using sklearn to implement my algorithm. In particular, I am using the GridSearchCV to optimize hyperparameter (for now, max_feautures and n_estimators), but GridSearchCV doesn't has a built in method to optimize for specificity. Further, specificity is a measure of statistical precision, and I would like to optimize for the value at risk in each purchase. I am considering using a cost matrix, but I am unsure on how to do this. Any advice welcome.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.