I want to create a "grey zone" for a binary classifier. Grey zone means, in this zone classifier should give the result "I don't know".
I denote classes with + and -.
I have good tpr , but not so good fpr. Hence I want to do
- Change thresholds separately for class positive and negative. For instance:
If instances has probability of belonging to class + higher than 0.9 -> +
If instances has probability of belonging to class - higher than 0.7-> -
Rest is grey zone, for instance 0.5 probability for class +, 0.5 probability for class - -> grey zone
How should I set up thresholds for two classes, would ROC threshold help?
Thanks in advance.