Classifying sequences of handwritten digits What's the best approach for recognising a sequence of handwritten digits?
EDIT:
By this I mean a program that might, for example, take this image as input, and output: 123.
Initial idea:


*

*Separate digits

*Recognise them individually using a multilayer feed forward neural network
My thoughts are that separating the digits could be tricky (if they overlap), and that perhaps there is a better approach using a recurrent or convolutional neural network?
 A: Here's one idea:


*

*Pick a size of image that a single digit will be in, and double that size for two digits, triple that size for three digits, etc. Blank space is okay, I just think that having a consistent size will be helpful. You could use something like n*28x28, where n is the number of digits. This allows you to use MNIST and randomly chain digits together.

*Train a neural network (NN) $N_1$ that is really good at recognizing a single digit.

*Make a NN $N_2$ that takes in an image containing two digits, and outputs two images. Feed these images into two copies of $N_1$, and see how good they did. Then "train" the entire network according to how accurate those outputs are, except you only modify $N_2$.

*Make a NN $N_3$ that takes in an image containing three digits, and outputs three images. Then train $N_3$ like we trained $N_2$ above, except using three copies of $N_1$. Alternatively you could use one $N_2$ and one $N_1$, I don't know which would be better so that's something to play with.

*Make a NN $N_4$ that takes in an image containing four digits, and outputs two images. Then train $N_4$ like we trained $N_2$ above, except using two copies of $N_2$ instead of two copies of $N_1$.

*Repeat this process until you have neural networks that can recognize, given any fixed number of digits, what those digits are.

*Create a dataset of images of size 2*28x28 containing randomly one or two digits.

*Train a neural network $C_2$ to be given these images, and correctly output one if there is one image, and two if there are two images.

*Create a neural network $C_4$ that is given 4*28x28 images, and outputs two images which are sent into $C_2$, and then $C_4$ is trained accordingly.

*Repeat this process until you have a neural network $C_k$ that can be given any image up to some large size n, and output how many digits it has. This k should be larger than the number of digits you ever expect to be given. Also you could totally start with a $C_{10}$ and then a $C_{20}$, the scaling of these is variable and so you can play with that to see which does  best/which takes less time. Also, it might be helpful to pre-process the
image before sending it into your network by removing the whitespace, then putting it into the nearest k*28x28 $C_k$.

*Now send any image into your neural network of size $C_k$, and it will output how many digits there are. Using this data, feed the image into the appropriate $N_i$, and this should give you the correct result.


This might not work at all, I don't know, but I suspect that this idea of using neural networks recursively what you're gonna need.
Also of course neural networks aren't the only option, there are probably other magical stats things that might work better for separating the images or doing any of these other tasks. So that's something to play with as well.
