2
$\begingroup$

I'm working with unbalanced class databases and I'm wondering what's the best practice to retrieve the best threshold cutoff value using roc curve ?

I want the best cutoff to maximize my F1 score.

I already have a roc curve I'm wondering what the best practice to generate the best threshold cutoff.

Improve this question
$\endgroup$
1
  • $\begingroup$ Slippery slope of a question... Please read this entry. If you are using R and want to calculate the values suggested here, I have it summarized for different packages in my notes here. $\endgroup$
    – Antoni Parellada
    Commented Sep 9, 2016 at 15:21

1 Answer 1

Reset to default
3
$\begingroup$

For any input (threshold) you have one output (F1 score), so, you can try to do a grid search, where you try every possible threshold from 0 to 1 in grid (say, seq(0,1,by=0.01)) and see which number maximize the F1 score.

In addition, the finding the best threshold can be also viewed as a one dimensional optimization problem (without using gradient). You can try optimize in R. Details can be found here. The difference between grid search and optimize is optimize is using a "smarter way" to search, e.g., if we see worse results we will not continue on that direction.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.