2
$\begingroup$

I am working on a multi-class classification problem. I was wondering if there is way to decide if a test sample could not belong to any of the classes in the training set? Simply put I want to classify if the data point belongs to an unseen class - 'others'

For now, I am thinking of extracting probabilities of how much each data point belongs to each class. If these probabilities are skewed (i.e. are weighed highly for some classes and very low for majority of classes), then it belongs to one of the train set classes. If these probabilities are uniform (say almost equally distributed) then it belongs to an others class. But obviously this method might give bad results and I would also need to empirically set the thresholds for uniformity and skewness.

$\endgroup$
2
$\begingroup$

This sounds like Outlier Detection to me. You should consider Unsupervised (Hierarchical Clustering) and Semi-Supervised methods.

For example, you could use One-class classification. Label all of your training as 'Class1' and see if the classifier assigns it to 'Class1' or 'Other'.

Using Python and data with some kind of distance function, have a look at:

If you want to use the technique you mentioned, here is a slightly different approach to get better results: Create One-vs-All classifiers. For each binary classifier, the class '1' is fitted against all the other classes '0'. Instances that never made it to class '1' for all classifiers can then be labeled as 'Other'.

http://scikit-learn.org/stable/modules/generated/sklearn.svm.OneClassSVM.html

Yes, I like SVMs.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer. But one-vs-all is a very brute Force approach. Aren't there any more formal methods to see if the data sample doesn't belong to any of the training classes? $\endgroup$ – silent_dev Sep 12 '17 at 12:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.