1
$\begingroup$

First question here, I am new to machine learning and wanted to understand the following: I used decision trees, boosting to classify fraud users and I am getting average precision around 60% on my test set. What does it signifies and how well am I able to do the classification.

$\endgroup$
1
  • $\begingroup$ Acceptable for what purpose? $\endgroup$
    – Sycorax
    Commented Jul 20, 2018 at 23:19

1 Answer 1

3
$\begingroup$

Precision and recall numbers depend on the actual probability of fraud in your data. Generally fraudulent cases are very small in number => Precision might be low, but recall should be high.

Acceptable or not can only be defined by measuring the impact of the results in real-life. I present two examples:

  1. Let's say for your business the goal is to avoid fraud at all costs, then one would strive to achieve as high recall values as possible. But if it matters to you that you don't predict non-fraud cases as fraud, maybe because it hurts customer-sentiment, precision is more important.
  2. Try quantifying the loss due to misclassification. Lets say a False Positive(hurts sentiment) and a False Negative(allows fraud) cause you losses of $l1$ and $l2$. As you strive to improve one of these, the other might get impacted. Thus, you have to exercise the trade-off well in order to get the right model.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.