Classification with unknown class I would like to know how to train a classifier that is able to recognize a couple of classes and records that do not belong to any of the existing classes.
Let's say there are 2 classes:


*

*High

*Low


But it can happen that some records do not belong to either of those classes, so we need a third state, like not recognized or other:


*

*High

*Low

*Other


How to train such a classifier? I'm guessing that this question could also be related to anomaly detection, since you identify examples that differ from what is expected.
 A: Yes, looks like anomaly detection problem. What you could also try is to generate artificial samples for your third class and train your model using them. Of course, the other question is how you generate it. But this highly depends on problem you solve.
A: This depends on your classifier. Any method that assigns weights to each class and then use a decision rule could be modified to your method. For instance, a random forest typically uses majority voting. Say you have one with 1000 decision trees. You could modify the method so as to use majority vote only if one of your classes is predicted by at least, say, 600 decision trees, otherwise output "Unknown" if the vote counts are too close, i.e. both are between 400 and 600.
A: Probably you don't need to change your classifier at all, just the way how you interpret results. E. g. if you have 2 classifiers and first one predict 10% probability that training sample belongs to it's class and the second one predict 9% probability, then it seems that training sample doesn't belong to any of that classes, so you can just label it as "other".
