In a binary classification scenario, is it a sound approach, if I built two separate classifiers which are each trained only on a single class (positive and negative strictly separated) -- thus completely overfitted and biased --, then used these two classifiers to calculate the probability for a new data item to belong to one of the classes and finally use the odds of the probabilities to decide to which class the data item belongs?

I think this is somehow related to ensemble learning or one-class learning, but I'm not sure if the proposed approach is a good one.


1 Answer 1


Don't do that. Binary classifiers work the way they do for very good reasons. If you want to model the difference between two classes, use both, rather than training 2 separate models that don't learn what separates the classes.

  • $\begingroup$ Yeah, this I also what I learned. But why is this a bad approach? If both are totally overfitted, they are "biased expert", and the one that has the "stronger arguments" wins. At least this was the idea. It's not theoretically founded, but the idea itself doesn't seem to stupid to me. Maybe if someone has some strong mathematical arguments or at least can point me to some literature. Anyway, I guess I just try and see how the results are :) $\endgroup$
    – Frank D
    Feb 10, 2015 at 22:14
  • $\begingroup$ Conceptually: if I ask you to describe the difference between A and B and I only give you examples of A, can you give a good answer? $\endgroup$ Feb 11, 2015 at 7:27
  • $\begingroup$ No, I can't. But the proposed approach is conceptually more like this: If I'm an expert for A and you give me an arbitrary example, I can tell you if it is a good example for A. The same for B. Then I have the two estimations of two experts and the one that is more certain about his opinion wins. $\endgroup$
    – Frank D
    Feb 11, 2015 at 21:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.