1
$\begingroup$

I have implemented a machine learning algorithm (say, algorithm A) to perform binary classifications of fraudulent and non fraudulent clients. I am already using other algorithms (say, B and C) to perform this classification and I want to see which of the three is the best one. What would be the appropriate experimental setting to do so?

So far, I have been using accuracy and recall as my main measures of model performance. In order to do this, I have needed to know which are false positives (clients that were labeled by the model as fraudulent when they actually were good clients). When the model is actually implemented, however, I won't be able to identify false positives unless I have a control group. Isn't this right?

How can I know which of the three algorithms is working, if any? Should I used ANOVA? If so, what variable should I perform ANOVA in (maybe the mean amount of money that was "rescued" from fraudsters)? What are other alternatives?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.