OCR model with TensorFlow

I'm just getting started in Machine Learning and I'd like to implement a simple OCR.

I've played with the MNIST dataset, but it was classification over a finite number of classes, and one character at a time (so no positionning/ordering issues). What kind of model should I look into to build a simple OCR ? How does character location in an image is usually treated ? For example, if the image contains a 9 and a 1, which techniques will allow me to know if it's 91 or 19 ?

There are many steps before we come to the "recognizing MNIST digit" step. And after recoginizing indivisual character there are also many steps to follow.

Let's assume we are performing a complicated tasks to parse the address information from a image of letter (image comes from here).

The steps are (and these steps are already over simplifed)

• Recognize the text area (seperate address info from stamp and other pictures)
• Segmentation for indivudual character
• Perform OCR on each character
• Combine all the recognized characters and perform semantic understanding from text

Your excise on MNIST digits recognition is only the third step. And your questions about differenciating 9 and 1 vs 91 is the fourth step.

In sum, understanding all the texts from an image is a very complicated task. Each step can be very complicated. And there are many pre-processing and post processing in each algorithm, (For example in the example letter, the sender and receivier's address are not in same size, after detecting each text erea, we need to re-scale them.). In addtion, it can be thousands of specific algorithms detecting the text area step alonge, also for segmentation step.

What you experienced, is just some toy example on a simplified step with very clean and well pre-processed data. In real world, things are more complicated and many similar building blocks are needed.

• Finally a realistic explanation on computer vision; multi step problems, even for "deep learning". – PascalVKooten Jan 31 '17 at 20:42
• @PascalvKooten i think the advanced software do too much things for us, and people would more likely to think the black box "machine learning model" can do everything with some simple API. – Haitao Du Jan 31 '17 at 22:12