I've built a (deep learning) model for a binary classification problem. For this problem I'm interested in sub-dividing my data set to maximize performance (as measured by precision) by essentially throwing out hard-to-predict samples. This would leave a smaller set of easier-to-predict samples with higher precision. How do I automate this?

For example say I'm predicting whether images are a hot dog or not (sorry). My overall performance as measured by precision is Y. If I pre-segment images into those with, say, key hot dog colors (red and yellow) and those without, my precision could be >Y for the images that pass the filter. I could set a lower score threshold (/ higher tolerance) for pass images and a higher score threshold (/ lower tolerance) for non-pass images. This would reduce the recall but boost the precision.

Is there a technique that automatically boosts precision by filtering in this manner? I.e. by varying score thresholds on different sub-populations? For my application, even reducing the population by 10x or 100x to a group of "sure thing" predictions would be beneficial.

Edit: Techniques like bagging or Adaboost might be the answer but AFAIK these methods don't explicitly try to remove samples but just adapt the model to sub-populations. Not sure if this is the same thing. Any insight?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.