1
$\begingroup$

I've built a (deep learning) model for a binary classification problem. For this problem I'm interested in sub-dividing my data set to maximize performance (as measured by precision) by essentially throwing out hard-to-predict samples. This would leave a smaller set of easier-to-predict samples with higher precision. How do I automate this?

For example say I'm predicting whether images are a hot dog or not (sorry). My overall performance as measured by precision is Y. If I pre-segment images into those with, say, key hot dog colors (red and yellow) and those without, my precision could be >Y for the images that pass the filter. I could set a lower score threshold (/ higher tolerance) for pass images and a higher score threshold (/ lower tolerance) for non-pass images. This would reduce the recall but boost the precision.

Is there a technique that automatically boosts precision by filtering in this manner? I.e. by varying score thresholds on different sub-populations? For my application, even reducing the population by 10x or 100x to a group of "sure thing" predictions would be beneficial.

Edit: Techniques like bagging or Adaboost might be the answer but AFAIK these methods don't explicitly try to remove samples but just adapt the model to sub-populations. Not sure if this is the same thing. Any insight?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.