1
$\begingroup$

I'm trying to build an algorithm for doing named entity extraction. It goes like this. There is a large set of text documents [communications], from which specific information has to be extracted. The information to be extracted is date, process names, terms like price, percentages, organization names, etc. It looks like standard nlp problem. But one problem is documents can vary in size and format. It can range from one line to text plus tables or couple of paragraphs with other discussions. My idea is that, if I build a multi-label classifier, with large enough data set, and classify documents, where each class represents particular type of document, for each type of which, there will be specific named entity and information extraction codes.

In the situation I'm right now, it's not possible for me to gain access to real data. I've only a few samples documents.

My question is this -

  1. Is my approach okay? Currently, I'm using python+nltk library.
  2. To train classifier, as I don't have large data set, I'm thinking of generating test data, by writing random text generator, based on the sample emails I have. If I do like this, won't I be fooling my classifier? Is there a fundamental problem with this approach? What should I do, as I don't have access to real data?

Any pointers will be helpful. Thank you!!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

  1. It looks ok but you will have problems scanning tables out of documents since table structure gets lost in PDFs.

  2. Yes there is a problem with randomly generated data and it is that it assumes the distribution of your samples and will weigh words differently than how they actually occur. Also words that are rare will skew your classifier and how would it handle with words that are not found in the training set?

Improve this answer
$\endgroup$
4
  • $\begingroup$ Hey thanks for the answer and link! I think i was not clear in my question, when I said documents, most of them are like html/email, not PDF. So, I can handle them, but my concern was format - like sometimes, it may be table, some other times, it may be white space elements like next line or tab. Now, for the specific problem, target named entity are few(which we'll have to identify)I Can use regex+lookup table but the problem is people can write in same word different ways, and sentence structure can vary, with spell mistakes/shortforms..I'm Trying lots of variety in random generated data.. $\endgroup$
    – tired and bored dev
    Dec 30, 2014 at 6:59
  • $\begingroup$ And how serious such influence of randomly generated data can be? Is there anyway I can reduce such an effect and make it look like it's been trained for real data? $\endgroup$
    – tired and bored dev
    Dec 30, 2014 at 7:42
  • $\begingroup$ If the data is integrated with a table it would be difficult to parse it out. It might be possible with Regex but I think you will get a lots of false positives. For your spelling the same words thing check for Lexical resources in NLTK. For your second question I am not sure how you randomly generate data so I cannot say. $\endgroup$
    – ccsv
    Dec 30, 2014 at 8:00
  • $\begingroup$ Thank you again! Yes. Data is integrated in the table (html/white space like tabs/line break). So that's one difficulty. Let me see how it'll turn out. Also, I will go ahead build multiple sets of randomly generated data sets.Basically,I've some typical documents,so I created a schema for documents,& from sets of phrase/sentence segments,I'm creating random data sets.So if I can do this for multiple independent random datasets successfully,I can then prove it on real dataset. related- stackoverflow.com/questions/10526579/… $\endgroup$
    – tired and bored dev
    Dec 30, 2014 at 8:15
1
$\begingroup$

I think you might wish to consider using a rule based matcher as you don't seem to have a large set of training data. Using some rules can provide you with a good baseline as well if you don't have any other results to compare your approach to.

For example I can recommend spacy's rule based matcher: https://spacy.io/usage/rule-based-matching

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.