2
$\begingroup$

I am training an NLP classifier that maps input sentences to 1 of 50 categories.

The model is a CNN language model, in which each input example is a 2d tensor of sentence length by word embedding vector size.

Every example in the test set is not present in training. I achieve the best test score on the data when regularizing with very high dropout (> 0.9). I can train this model so that test approx = train score after training.

However, the predictions from this model are all very high confidence. Furthermore, there are many predictions with confidence = 1.0. How can this be possible on unseen data?

dropout regularized model

If I use L2 regularization instead the problem goes away, however, the model cannot reach the same test score.

L2 regularized model

Could there be any valid theoretical reason why this might occur? Any ideas to solve the problem?

It could be that this is just a very, very good model. The data contains many annotation errors, which explains the high confidence incorrect predictions. There are low confidence predictions, just not as many as I would expect to see... But the confidence = 1 on any test data seems wrong to me.

$\endgroup$
1
$\begingroup$

Firstly, be aware that the outputs of a normal neural network have nothing to do with the confidence of the predictions. If the output of the softmax layer is nearly 1 for some class, that does not automatically mean that the network is almost certain of the prediction. Softmax only normalizes outputs so they sum up to 1, but it does not provide any confidence about that prediction.

That said, I think you simply did a good job training the model for the task. Since you have not given any details about the dataset, I guess it might simply be that the task is relatively easy. A well regularized model should perform comparably good on training and testing data, and 0.9 dropout rate is very strong regularization.

If you are suspicious, try inspecting the data. Make sure the vector representation of validation set sentences is not the same as the sentences in the training data.

$\endgroup$
  • $\begingroup$ Interesting. Do you not think outputs of 1.0 for a single class are worrying on the test set? This would infer that an example produces values of 0.0 at every other output node in output layer. $\endgroup$ – Ollie Feb 5 '18 at 21:39
  • 1
    $\begingroup$ This paper also seems to elaborate on the problem: arxiv.org/pdf/1706.04599.pdf $\endgroup$ – Ollie Feb 5 '18 at 21:40
  • $\begingroup$ Are the predictions really exactly 1.0? That would be unlikely... From the histogram it seems they are rather in the interval 0.97-1.0. That can happen. $\endgroup$ – Jan Kukacka Feb 5 '18 at 21:45
  • 1
    $\begingroup$ @JanKukacka Do you have a reference for that bolded part? Since you are maximizing the multinomial likelihood, the softmax probabilities seem like they should indicate some confidence in the classes. $\endgroup$ – Matthew Drury Feb 5 '18 at 23:36
  • 2
    $\begingroup$ @MatthewDrury it is quite common misconception. See e.g. this blog by Yarin Gal mlg.eng.cam.ac.uk/yarin/blog_3d801aa532c1ce.html or this article: arxiv.org/abs/1707.07013 $\endgroup$ – Jan Kukacka Feb 6 '18 at 6:05
1
$\begingroup$

To elaborate on this problem. This is know as a poorly calibrated model, where the network outputs cannot be interpreted as predicted probabilities. You can visualize this by plotting accuracy per confidence bin for the test set. A well calibrated model will resemble the left hand plot, whereas a badly calibrated model will resemble the right plot.

enter image description here

Indeed when plotting this for my model above, it was very poorly calibrated.

This paper investigates causes and techniques to solve the problem:

https://arxiv.org/pdf/1706.04599.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.