Given a classifier, that was trained on a data having two classes e.g. class
red and class
blue. The training samples are hand picked and only contains those two classes.
After training the classifier should classify some samples, but it turns out that those samples contain the classes
blue and a third one
red+blue) e.g a class that contains features of both classes. The classifier will correctly identify
blue but will degrade to a random guesser for
I try to give an example here (of course this is much simpler than in real life...):
assume that a sample has the features
B, which are binary fields (either
The training data looks like this:
A B Label ------------- 1 0 red 1 0 red 0 1 blue 0 1 blue
One possibility for the classifier (e.g. a tree) would be to have the following output:
If A == 1: red If A == 0: blue
Well, obviously this model is too general (but for this example I hope this makes no difference).
Now the classifier is tested on some samples:
A B Output -------------- 1 0 red 0 1 blue 1 1 red
Here is the problem.
The classifier will just output some "random" label for the third sample, depending on the training. If this oversimplified classifier would have looked at label
B, it would outputted
From other knowledge (e.g. a Gold-Standard) we can see that the third sample is actually neither
blue but our
The classifier is just too general (?) or not well-trained (?). Is this a Sampling Error, because the third class was not identified as such during training (because the data was not available in the training set)?
How is this problem called exactly? I'm searching for the term but can not find one...
What could be a countermeasure? Of course, one could add samples of class
magenta into the training set. Are there other possibilities?