I am working on a Named Entity Recognition (NER) project. Instead of using an existing library, I decided to implement one from scratch because I wanna learn the basics of how PGMs work under the hood. I converted the words in sentences into feature vectors. The features are manually picked by me, and I can only think of roughly ~20 features (such as: "Is the token capitalized?", "Is the token an English word?", etc.). However, I've heard good NER algorithms represent tokens using way more than 20 features, sometimes hundreds of features. How do they manage to think of so many features? Are there any recommended best practices in feature construction?

  • $\begingroup$ is your question about the thought process that goes into feature selection, or about what other additional features for a NER algorithm might be? $\endgroup$ – David Marx Aug 6 '13 at 20:13
  • $\begingroup$ Hi David, I think I need to know more about what other additional features for NER, and also what are some common approaches to find these features. Thanks $\endgroup$ – xiaoyao Aug 6 '13 at 20:29
  • 3
    $\begingroup$ One place to start: you might consider comparing the kinds of features you've developed with the kinds of features in the Stanford NER library (reference slides 10 and 11): nlp.stanford.edu/software/jenny-ner-2007.pdf $\endgroup$ – David Marx Aug 6 '13 at 20:56
  • 2
    $\begingroup$ Often times the huge numbers of features can come from sets with extremely high cardinality, like the vocabulary in your document collection, the part of speech, and so on. It's also fairly common to use features from neighboring words, so it's not necessarily the case that people are thinking of lots of unique features focused only on the target token. $\endgroup$ – lmjohns3 Aug 9 '13 at 21:21

Indeed to have an efficient NER you need a lot of features. If you start from scratch (what I did first as well) it's really hard to figure out what features could be used other than obvious ones you mentionned. But what really boosted my scores on the one I built was introducing context grammar, tagging and parsing the sentence and use it. You can also add a word vectorial representation. Last, it seems important to add some word-specific features when you encounter difficult cases (e.g. the New-York Times, you can add a feature specially for this). You should also add big dictionnaries and have dimensions of your feature vector that tell if the word belongs to a specific dictionary...

Good luck, it's a really hard problem to get a good NER and building feature is most of the time linguistic knowledge more than mathematical ones!

  • $\begingroup$ m to the v to the a $\endgroup$ – mic Jan 22 '16 at 21:25
  • $\begingroup$ +1 for MVA! Guess my nickname reveals my id quite easily though =) $\endgroup$ – Vince.Bdn Jan 22 '16 at 21:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.