I have millions of user generated document that contain smartphone specifications. I want to get certain properties from those documents, e.g: OS, display resolution, processor, RAM, camera resolution. Document sample:

Dimensions 142.1 x 71.8 x 7.9 mm (5.59 x 2.83 x 0.31 in)
Weight 146 g (5.15 oz)
SIM Optional Dual SIM (Micro-SIM, dual stand-by)
Type Super AMOLED capacitive touchscreen, 16M colors
Size 5.0 inches (~67.5% screen-to-body ratio)
Resolution 720 x 1280 pixels (~294 ppi pixel density)
Multitouch Yes
OS Android OS, v5.1 (Lollipop)
Chipset Qualcomm MSM8916 Snapdragon 410
CPU Quad-core 1.2 GHz Cortex-A53
GPU Adreno 306
Card slot microSD, up to 128 GB
Internal 8 GB, 1.5 GB RAM
Primary 13 MP, 4128 x 3096 pixels, autofocus, LED flash
Features Geo-tagging, touch focus, face detection
Video 1080p@30fps
Secondary 5 MP, LED flash

From that document, I want to have this:

OS: Android
Display resolution: 720 x 1280
Processor: Quad-core 1.2 GHz Cortex-A53
RAM: 1.5GB
Camera resolution: 13MP, 5MP

Please note that my document will not always that good, in fact it is one of the most structured document. What is the name of problem that I want to solve? Is this a 'named entity recognition' problem?

  • $\begingroup$ This looks like a slot filling problem. It is very similar to named entity recognition. $\endgroup$ – Aaron Apr 21 '16 at 6:18

This task can be formulated as a slot filling problem. People have been using CRFs or RNNs for the model. It does have a lot of similarity to named entity recognition. Anything you would use for NER would probably work for slot filling too


There is not much literature on semi-structured document parsing. You can approach it with Named Entity Recognition but all the pre-trained models such as Conditional Random Fields (CRFs) won't suit your needs since they aim at PEOPLE, ORGANISATION, and some other basic classes. Building a training set for your kind of data to train such models is very costly (time consuming mostly). And if you try to automate the process to annotate and build your training set, then you have solved the problem...

This problem is really non-trivial (at least to solve with a generic approach), and is overlooked in the literature.

I know some guys who are specialized in that. Their examples are much similar to yours. Maybe you can drop them a mail to learn about some techniques to solve this problem: their website http://www.scriptminer.com/


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.