2
$\begingroup$

I have a two-class classification problem with n-dimensional data. I would like to train a classifier (preferably but not necessarily linear) with 100% positive predictive value. In other words, I want the model to completely avoid one of the classes. For this application a low-ish sensitivity is OK as long as PPV is ~100%.
Do you have any suggestions of good techniques to use? Thank you!

$\endgroup$
3
$\begingroup$

If by technique you mean classification method (logistic regression, classification tree, ...), you can use any of these methods to obtain the result you want. Each method usually has a build in cost-function that you can adjust to obtain your desired results. All of these methods end up as being equivalent to building an ROC curve and choosing which point on that curve you want.

Usually this is done automatically for you so you might not be aware that there is a tuning parameter that should be changed if you have an explicit cost function.

Thus logistic regression usually uses the classification split at 0.5 probability, but based on your cost function you can change this to obtain the desired sensitivity/specificity. Most standard statistical packages will contain a post-estimation command that you can use after you build your regression model to provide sen/spec/ppv/npv for all the possible cut-point from 0 to 1.

Perhaps it is worth noting that the cost function is rarely expressed at "goal of 100%PPV" but often as a ratio: the cost of false negatives/cost of false positives. In your case this ratio is low. The cost of a false positive >> cost false negative. But estimating this ratio you can give a more precise measure of your cost function.

Edit: what i have called the cost function above is usually called the "utility function" in texts.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Thanks for the response Charles! The problem with the cost sensitivity and threshold tuning approaches you described is that, while they are good ways to tweak a model's PPV, they are not very effective ways to maximize the PPV. This is because they work by shifting the position of the model's decision boundary, but not its orientation. For maximum PPV we often need a completely different decision boundary than the one produce by the standard model training process. I don't know if there's a "right" way to tackle the problem of maximizing PPV. (...) $\endgroup$ – Eric C Jul 2 '14 at 8:40
  • 1
    $\begingroup$ (Con't) But I recently published this paper in 'Pattern Recognition' describing the problem and one approach that seems to be effective. Let me know if you have any other ideas, Thanks! (added by moderator when converting answer to comment) $\endgroup$ – chl Jul 2 '14 at 9:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.