I have data that is split into three classes (A, B and noise). The data amount is around 10000 samples, and A and B is only less than 5-10% of data. What is the best approach to handle this situation and what ML methods are most suitable for it?

The misclassification of noise (treating noise as A or B) and treating A as B (or vice versa) is more important to avoid. Treating A or B as noise is not so important error unless we have some correct A and classifications.

In this case, I'm also thinking of making multistage system (first filter noise using binary classification) and then classify data into A and B classes. Will this approach be more useful?


Sounds like a multi-stage approach - as you already suggested - would make sense in this setup. The first stage would filter noise, the second stage would distinguish between A and B.

The advantage of such a split is two-fold:

  • First, each stage can be tuned individually. This means that you can tune for example the FAR for noise independently of the later A/B separation (e.g. decrease the FAR at the cost of increasing the FRR of A/B, which seems not so problematic for your case). Further, if A and B share some properties compared to noise, this might also make the separation problem itself easier.

  • Second, the class imbalance between A/B and noise can be accounted in the first stage and does not directly influence the second stage.

Which models you use for the first and second stage will depend on your problem and data - you most likely want to evaluate multiple to see for possible differences in performance.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.