I am constructing a training dataset for a classification problem with three categories: A, B and C. Once estimated/learned, I want to use the classifier to identify observations of types B and C in new data.

My problem is that almost 90% of observations are in A, so I get very few training examples of B and C and thus provide little information to the algorithm about how B differs from C.

Is there a recommended way for handling this? For example, can I throw out cases that are highly likley to be A before I construct the training dataset?

  • 1
    $\begingroup$ Try training multiple models on randomly selected subsets of those A's that are part of your training data, and averaging or combining the results of the multiple models. Another thing : break it into two stages. Identify a class (not A), followed by a differentiation between B and C. $\endgroup$ Aug 9, 2017 at 19:40
  • $\begingroup$ The two-stage approach you are suggesting is what I had in mind, too. Are you aware of any references on this or any documented applications? $\endgroup$ Aug 9, 2017 at 19:45
  • $\begingroup$ I believe the technique is called "bagging." It's short for bootstrap aggregating. The second method is nothing special. $\endgroup$ Aug 9, 2017 at 19:49
  • $\begingroup$ Thanks, but I meant the second suggestion, i.e. the idea that I should first identify "not A" observations and then classify further in a second stage.. $\endgroup$ Aug 9, 2017 at 19:51
  • $\begingroup$ The intention behind the second suggestion is that models like binary situations, and more so when the number in one class is closer to the other. $\endgroup$ Aug 9, 2017 at 19:53

1 Answer 1


This is an imbalanced class classification problem.

One common approach would be undersampling or oversampling - depending on the amount of training data available for you.

You can read more in 8 Tactics to Combat Imbalanced Classes in Your Machine Learning Dataset

  • $\begingroup$ Thanks. I think I was not clear. The problem is that I need to manually code the training data and it is not feasible for me to get a large enough number of observations in B and C within the training dataset unless I can somehow remove observations of A in a first step. Does that make more sense? $\endgroup$ Aug 10, 2017 at 20:19
  • $\begingroup$ I still see this as an imbalanced class problem. Undersampling or oversampling is actually a method of "manually coding" the training data. If the data is too skewed maybe think of this as an anomaly detection problem (as the article suggests), identify B and C as outliers and then train a classifier to distinguish B and C. That is just a one of the many things you can try though. $\endgroup$
    – akilat90
    Aug 12, 2017 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.