Classification with increasing number of classes

Question

I have dataset with large number of labelled data (say that there are k classes). I have also another, much smaller dataset with unlabelled data that I want also to be labelled. The problem is that in the second dataset, number of classes need not to be the same as number of classes in first dataset (more precisely, the there may be more classes). In other words, some objects from unlabelled dataset can be classified to one of k possible classes since they are "too close of them" but some objects should be clasified to new classes.

Real world example: In some applications, number of classes can increase with time. For example, number of known species increases with time, so when I see any unknown plant in nature, biologist can either to classify it or to say that this is new discovery (new class).

Is there any technique that deals with this type of classification? Any help is appreciated.

Answer 1

This problem is sometimes called Open set recognition, or classification. There was a recent survey on open set recognition on Arxiv https://arxiv.org/pdf/1811.08581

The problem is also called zero shot learning https://en.wikipedia.org/wiki/Zero-shot_learning when the data points are images (I think)

Answer 2

It seems it is a clustering problem - if you don't know what and how many classes will you have, then unsupervised learning is for you. For example, if your trained classifier is very confident on some training example, then you conclude that it belongs to a known class. And if your classifier is unsure (say, it predicts probability less then 30% for EVERY existing class), then you conclude that you have encountered some previously unknown class. In such case you can use unsupervised learning on those unknown classes or label those samples manually, introducing some new classes.

Answer 3

You may treat all known classes as one class, and run some outlier detection / novel detection algorithms. For example, one class SVM

Therefore we may have hierarchical classifiers, the first one to check if the data is some new species, if no, then second classifier will categorize data to $k$ known classes.