2
$\begingroup$

I had this problem from a long time. I have small dataset with about 1000 data points. The data is labeled as 1 or 0 (i.e. binary classification). In other words if the product is defective it is marked as 1 and 0 otherwise.

The features of the data are the product properties (such as height, width etc.). Since my dataset is very small, I initially performed 10 fold cross-validation to perform my classification. Now that my classification part is done, I encountered another problem.

The problem is to rank the most defective products first (i.e. a priortised list where the top contains the most defected items, so that the actions can be taken in that order).

I want to use my same features to do the ranking. For this purpose, I am considering the prediction probability of class 1 of each data point when it is in testing fold of 10-fold cross validation (i.e. using predict_proba in sklearn python). Then I sort all the 1000 data points based on this probability to get a priortised ranking list.

My concern is whether what I am doing is correct? If not, what are the other options that I can try?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Your approach is not wrong, though I would suggest a different interpretation. Those cases with higher probability values are ‘more likely to be defective, given the data’ rather than those cases are ‘most defective’. It is binary classification after all, and your training data either was or was not defective.

Improve this answer
$\endgroup$
6
  • $\begingroup$ thank you for the great answer. Do you have any suggestions to identify the most defective items? :) $\endgroup$
    – EmJ
    Dec 4, 2019 at 6:37
  • 2
    $\begingroup$ In binary classification there is no concept of ‘most defective’ just ‘most likely to be’ defective. To get what you want would require that your training data include a real-valued assessment of ‘amount of failure’ first, then you wouldn’t use binary classification but one that outputs real valued predictions $\endgroup$
    – HEITZ
    Dec 4, 2019 at 6:40
  • $\begingroup$ That is a great suggestion. I do have a percentage value for each data point that denotes how much it was defected. Like 80%, 90% or 0% etc. I did the labelling of 0 and 1 based on this. Since, I am having these percentage values, how can I use them to rank the predictions? Looking forward to hearing from you :) $\endgroup$
    – EmJ
    Dec 4, 2019 at 6:44
  • 2
    $\begingroup$ You probably do not want to recode to 1/0 then. Your cut point is arbitrary and you throw away information. You’ll want to start trying some regression based techniques, and you’ll likely need to transform your y variable as percentages behave differently. Then your model will output amount of defect. $\endgroup$
    – HEITZ
    Dec 4, 2019 at 6:48
  • $\begingroup$ Thanks a lot. It is good to know that what I did previously was right. I will try the regression model :) $\endgroup$
    – EmJ
    Dec 4, 2019 at 6:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.