With TensorFlow I've successfully built a neural network that recognises MNIST numbers at 99.2% accuracy. The last layer of the network is the softmax layer, which does output percentages. The AdamOptimizer is used for training and cross entropy is used as the error function.

However, I'm not quite satisfied with the results. I took a closer look at the raw images. The first image (MNIST #449) looks like neither 5 nor 3 and was labeled 5, and the network outputs 3 with 67% confidence and 5 with 32% confidence. This seems acceptable for me, but another image (MNIST #1112) does look like 4, but the network outputted 4 with 6.62% confidence and 8 with 93.36% confidence.

Is there a better approach to make the network output probabilities? For example, maybe make the network detect which pictures are biased, and assign them with a higher entropy?

  • $\begingroup$ A softmax final layer is the usual method of having output probabilities. Based on your description of the incorrectly classified 4, it doesn't sound like an issue specifically with the final output softmax - rather that there aren't many other cases where a 4 of that shape appear and so it is closer in the parameter space to the network's idea of an 8. Have you tried augmenting your data, for example via shifts or rotations? Short summary here. It may give your network the additional information to distinguish that 4 from an 8. $\endgroup$ – n1k31t4 Feb 18 '17 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.