I am using nolearn with Lasagne to train a simple Multilayer-Perceptron (MLP) for the MNIST dataset. I get about 97% accuracy on the test set after training on the training set, which is a few thousand samples. Probably as good as it can get without using a Convolutional neural network (CNN).
I have created a simple paint program with pygame that lets me draw a digit and get it classified with my net.
Every time I want to classify a digit the following happens:
- Resize the draw surface to 28x28 (what they are in MNIST)
- Rotate and flip the draw surface to get correct orientation.
- Transform surface to a numpy array of pixels.
- Transform RGB to grayscale.
- Divide the array by 255 to scale between 0 and 1 (as what I did with the MNIST training set)
- Reshape array from 28x28 to 784x1 as what is expected of the net.
I have verified that my array is correct, in the sense that I have compared it against random training / test samples, and also written it out as a image to really be sure it is correctly transformed before using it.
I have tried to draw perfect digits, as well as possible and centered as possible with a good line width. I have tried to draw after samples in the MNIST dataset.
The problem is that I get very bad performance from my net of the drawn digits, way lower than expected. I can draw just one line in the middle and expect it to classify it as 1 with high probability, only to find out it classify it as a 7 or a 5 with mediocre probability. The inspiration for doing this comes from this video and clearly you can see how well it performs, though I think he does everything from scratch.
I have also tried different neural-network libraries but they perform the same.