# Which Object detection model will give the best result on images when the speed is not a problem for Text Images

I want to develop a model for cropping the equations from the Maths questions as people like me are struggling a lot for doing it manually for the research purpose. I want to know if we can do this? and if we can out of all the possible solutions out there for object recognition models, which one will produce the best results on Text images.

As there is tensorflow's object recognition API, RCNN, Fast RCNN, Faster RCNN, YOLO (v-1,2,3,4,5).

An if there is any other , please do suggest. What I want to do is to detect the gray areas of equations in this image.

Note: The grey region shown in the image is for just demonstrating. My actual images are simple cropped questions from books with with background and black letters (most of the books)

• Are math segments always highlighted with gray background? If yes, a simple deterministic image processing routine will do the trick. Commented Jul 22, 2020 at 11:17
• No no!!,! This is just to show what exactly I want to do Commented Jul 22, 2020 at 11:58

Note that there are two problems in this case: segmentation and classification. A neural net might be a solution for both steps in this case because you can easily generate zillions of labelled test images. Nevertheless, a classic approach should yield comparable results with much less efforts:

1. Use a simple page segmentation alorithm like runlength smearing or bounding box merging for segmenting the image into regions
2. Classify each region with an arbitrary classifier. You can use a NN on all normalized input pixels for this, but other classifiers like kNN should also work with gradient histograms as features (the gradients are computed on quasi grayscale images, which are generated from the onbit images by blurring). Gradient histograms were the state-of-the art features before the renaissance of neural nets.

Out of curiosity, I have tried out step one with the python library Gamera (gamera.sf.net) with the following code:

from gamera.core import *
init_gamera()

img = img.to_onebit()

img.remove_border()
segments = img.runlength_smearing()

# now you could process each segment (e.g. saving it to a file)
for seg in segments:
# do some stuff

# visualize the result
color_ccs = img.graph_color_ccs(segments)
color_ccs.save_PNG("segments.png")


The result looks reasonable to me(note that the colors only indicate the segmentation, with adjacent segments having different colors):

• Could you please give an example of this approach? And how can I crop using your idea? Commented Jul 22, 2020 at 13:22
• I do not understand. Waht do you mean by "crop"? And waht do you mean by "an example"? Example code? This will depend on the programming language and libraries at hand. Commented Jul 22, 2020 at 13:36
• I have added sample code for doing the page segmentation (it heavily depends on the pythin library Gamera, of which I am the lead developer). Is this what you wer looking for? Commented Jul 22, 2020 at 14:26
• I am looking to extract the portion of the code which can crop the equation from an image of question. For example if I put the program to a script, I want it to give me 2 cropped images consisting of both the equations seperately. Commented Jul 22, 2020 at 15:31
• My code above already solves the segmentation problem (write each segment to a separate file). Then you must only find a classification method. You write in your question, you are experienced with neural nets, so this is an option: generate a million sample images, store in a label file the classes of segments, use the above code for segmentation and train the NN with the labelled data. You cannot expect everything to be done by someone else for you ;-) Commented Jul 22, 2020 at 15:47