# Developing a classifier that can also flag anomalies

I am developing a classifier that can classify certain bags based on their picture. Testing it out in the real world, I realize that a lot of the time it gets passed examples that are not in the set of class labels. How would I mark these as anomalies rather than just misclassifying them as something within the set of labels? Should I switch to an unsupervised approach where I check the probability that it falls into an existing distribution then set a threshold to mark it as an anomaly?

Should I switch to an unsupervised approach where I check the probability that it falls into an existing distribution then set a threshold to mark it as an anomaly?

I would not do this. The threshold will be hard to calibrate, and you will have to observe the impact of modifying this threshold with modifications of other (hyper)parameters of your model.

Instead, I would suggest something more naive : did you try adding an extra class "anomaly" ? I guess your training set looks like:

bag1.png|classA
bag2.png|classA
bag3.png|classB
bag4.png|classB
bag5.png|classC


You may add an class anomaly, which corresponds to pictures that do not belong to your training classes

bag6.png|anomaly
cat1.png|anomaly

• I have tried adding an "anomaly" class but I find that it's hard to get a good representation of what an "anomaly" actually is. It could come from a completely different distribution than what the anomaly class is trained on. This resulted in the classifier classifying known classes as anomalies and anomalies as known classes which hurt its accuracy. Oct 25, 2017 at 9:41

Two things come to my mind:

1) If your classifier is linear then you can check if the prediction is not "close" to any other classes than to which it is predicted. If this is the case then it can be an example far away from the data-space and label that as anomaly.

2) Try building one-class classifiers for each of your classes and then whatever is not classified as in-class by them will be anomalies.