I am currently working on a project, where the problem statement is to detect handwritten text from a image of a particular form. As a pre-processing step I have extracted texts in the form of bounding boxes, and I have around 1500 images of texts extracted from the image form, out of which 50 of them are handwritten.

The problem is how do I now use these extracted images to train a classifier model which will classify the images as printed or handwritten text. For example what are the features to be taken into account to solve for this problem. I have gone though few papers, which suggests to take into account the following features: Deviation of width Density i.e the area of black pixels in the bounding boxes/total area of the bounding boxes Pixel distribution Now the problem is to extract atleast the above mentioned features from the bounding boxes.I have no prior knowledge of Deep learning. Any help will be appreciated. I am uploading the image and the extracted images, as well as the code to extract the texts from the images.

im_ns = cv.imread('C:/Users/arindam/Documents/Data/Image processing/IMG_20180921_111952.png')
gray = cv.cvtColor(im_ns,cv.COLOR_BGR2GRAY)
blurred_g = cv.GaussianBlur(gray,(11,11),0)

ret, th1 = cv.threshold(blurred_g,127,255,cv.THRESH_BINARY)
th2 = cv.adaptiveThreshold(blurred_g,255,cv.ADAPTIVE_THRESH_MEAN_C,cv.THRESH_BINARY,11,2)
th3 = cv.adaptiveThreshold(blurred_g,255,cv.ADAPTIVE_THRESH_GAUSSIAN_C,cv.THRESH_BINARY,11,2)

##Detecting horizontal Lines and removing them
th3_di1 = th3_di.copy()
hor = int(round(th3_di1.shape[1]/30,0))
hor_struc = cv.getStructuringElement(cv.MORPH_RECT,(hor,1))

bw_hor_er = cv.erode(th3_di1,hor_struc,iterations=1)
bw_hor_di = cv.dilate(th3_di1,hor_struc,iterations=1)

for i in range(0,bw_hor_di.shape[0]):
    for j in range(0,bw_hor_di.shape[1]):
        if bw_hor_di[i,j] == 0:
            th3_di1[i,j] = 255
        else:
            th3_di1[i,j] = th3_di1[i,j]

plt.figure(figsize=(20,25))
plt.imshow(th3_di1,'gray')

# perform a connected component analysis on the thresholded
# image, then initialize a mask to store only the "large"
# components
labels = measure.label(th3_di1, neighbors=4, background=255)
mask = np.zeros(th3_di1.shape, dtype="uint8")

plt.figure(figsize=(30,25))
plt.imshow(labels)


# loop over the unique components
for lab in np.unique(labels):
    # if this is the background label, ignore it
    if lab == 0:
        continue

    # otherwise, construct the label mask and count the
    # number of pixels 
    labelMask = np.zeros(th3_di.shape, dtype="uint8")
    labelMask[labels == lab] = 255
    numPixels = cv.countNonZero(labelMask)

    # if the number of pixels in the component is sufficiently
    # large, then add it to our mask of "large blobs"
    if numPixels > 8:
        mask = cv.add(mask, labelMask)


plt.figure(figsize=(30,24))
plt.imshow(mask,'gray')

# find the contours in the mask, then sort them from left to
# right
cnts = cv.findContours(mask.copy(), cv.RETR_EXTERNAL,
    cv.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if imutils.is_cv2() else cnts[1]
cnts = contours.sort_contours(cnts)[0]

# loop over the contours to make rectangles for the th3 image with gassian thresholding
for (i, c) in enumerate(cnts):
    # draw the bright spot on the image
    (x,y,w,h) = cv.boundingRect(c)
    #((cX, cY), radius) = cv.minEnclosingCircle(c)
    cv.rectangle(th3,(x,y),(x+w,y+h),(0,255),2)
    cv.putText(th3, "",(x+w+10,y+h),0,0.3,(0,255,0)) 
# show the output image
cv.imshow("Image", th3)
cv.waitKey(10000)
cv.destroyAllWindows()


##Extracting the bounding boxes
idx=0   
for (i, c) in enumerate(cnts):
    # draw the bright spot on the image
    idx += 1
    x,y,w,h = cv.boundingRect(c)
    roi = im_ns[y:y+h,x:x+w]
    #((cX, cY), radius) = cv.minEnclosingCircle(c)
    #cv.rectangle(im_ns,(x,y),(x+w,y+h),(0,255),2)
    cv.imwrite(str(idx)+'.jpg',roi)

Extracted Printed Text

Extracted Handwritten Text

Edited Actual document

closed as too broad by jbowman, kjetil b halvorsen, Peter Flom Oct 12 at 13:57

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Are you sure you are legally allowed to publish a photo of that contract on the internet? – Jan Kukacka Oct 11 at 13:44
  • Thanks for the heads up! Edited the doc. It shouldn't be a problem though. – Arindam Bose Oct 11 at 14:01