# (Deep learning) classification confidence

I have a model that is trained on n classes and has more than 95% accuracy on the test set.

The model is going to receive a mixture of images that are either from one of the n classes or from unknown classes. My guess / expectation was that in the latter case, the distribution of values on the last layer (softmax) would be more or less uniform. Then to spot an alien image, it would be enough to threshold the prediction layer.

Unfortunately that is not the case, and the model seem to express a very high level of confidence every time. Even worse (using keras):

>>> max(model.predict(np.random.rand(1, 299, 299, 3))[0])
1.0


So basically the model is 100% sure that random noise is one of the class.

How to detect the images that do not belong to one of the classes that the model was trained on?

Note: everything has been normalized and in Keras predict returns probability, not class numbers.

• It's hard to say without reproducible code. Did you normalize the training set? If yes, check that your random input has the same normalization. Otherwise, what does 95% accuracy mean with respect to 5 classes? Does that mean each class has a 95% True Positive rate? Jun 6 '17 at 21:17
• Yes, everything is normalized between 0 and 1. To answer your second question, it means that each class has 95% accuracy.
– meto
Jun 6 '17 at 21:47
• Are you sure that predict retuns probabilities and not a class prediction? Often python apis have a predict_proba method. Jun 7 '17 at 0:52
• Good point, but yes, predict return a vector with probabilities, not the class itlsef. in general you would have to do argmax to get the class. but my point is that the model was way too confident of the prediction
– meto
Jun 7 '17 at 3:48
• This is a well known problem in deep learning, see e.g. Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images
– sebp
Aug 31 '18 at 8:00