3
$\begingroup$

I am interested in recreating (for cost and data sharing discrepancies) a solution that machine learning solution companies like ABBYY have done. I would like to be able to take purchase order PDFs and convert the OCR text extraction to line item data.

Several features would need to be extracted and a .csv should look something like: (PO ID Number,Product/service ID: UPC/SKU, Product/ service description,Manufacturer/vendor, Etc.)

To clarify my question:

  1. Do I need to train a model for each feature I want to extract or are multivariate ML models for this?

  2. Is there a documented workflow for such a task?

  3. There are thousands of PO PDF's that could have many different vendors thus the format may not be the same.

I plan to use R to for this, but would eventually like to move into Microsoft Azure ML where I just use R in the pre-processing stack and run the algorithm/ deployed model with Azure so that either the client's or our dev team may find use in an application.

Improve this question
$\endgroup$
4
  • $\begingroup$ Do you have already annotated example for each information type you want to extract? $\endgroup$ – gdupont Aug 10 '16 at 21:53
  • $\begingroup$ By annotated you mean take the OCRed text and do something like part of speech tagging correct? maybe 1234123123//UPC? In this case I have not, but can do that. I would want to make the annotated training set from as many possible unique invoice/PO documents. My question here is: what triggers the algorithm to tag the number sequence as a UPC. Is it the fact that the literal text string "UPC #:" preceded the number or the fact that every UPC is n digits long? $\endgroup$ – Austin Przybysz Aug 11 '16 at 13:58
  • $\begingroup$ I meant do you have documents where you know the position of the features? For instance you have annotations telling that upc is on page 1 at position (x1,y1 ; x2,y2). If yes for all features, you do feature localization first and character recognition after. $\endgroup$ – gdupont Aug 11 '16 at 15:03
  • $\begingroup$ Ahhh thats an interesting way to go about this and probably so much more accurate. I need to get my hands on some documents first. My thoughts were to convert the whole OCRed text into a text file and have the algorithm parse one feature at a time based off recurring text pattern. Buuuut if the UPC appears in the same location every time you could "crop" that section of the pic out and run OCR only over that piece? Cool idea! My fear is that there are hundreds of formats but the 80/20 rule says I can get 80% of the spend from 20% of the suppliers. $\endgroup$ – Austin Przybysz Aug 11 '16 at 15:54
0
$\begingroup$

To answer the first question: having dedicated models for each feature to extract should be the best approach. Basically each model will try to distinguish between what you want for the feature (e.g. Product id) vs everything else. So it will ease the discrimination and each feature will have a more reduce vocabulary and probably structural prior for character level recognition.

In terms of workflow, it might be beneficial to have parallel pipeline for each feature to extract. In each pipeline, the first task will be to locate area of interest where it is very likely to find the target feature and the apply actual OCR at character level for extracting the content.

If you have a good OCR, you can also apply first a generic model on the whole content, use the output for coarse grain localization of the target feature and then apply your dedicated model for each located feature.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.