2
$\begingroup$

I'm new to Named Entity Recognition and I'm having some trouble understanding what/how features are used for this task.

Some papers I've read so far mention features used, but don't really explain them, for example in Introduction to the CoNLL-2003 Shared Task:Language-Independent Named Entity Recognition, the following features are mentioned:

Main features used by the the sixteen systems that participated in the CoNLL-2003 shared task sorted by performance on the English test data. Aff: affix information (n-grams); bag: bag of words; cas: global case information; chu: chunk tags; doc: global document information; gaz: gazetteers; lex: lexical features; ort: orthographic information; pat: orthographic patterns (like Aa0); pos: part-of-speech tags; pre: previously predicted NE tags; quo: flag signing that the word is between quotes; tri: trigger words.

I'm a bit confused by some of these, however. For example:

  • isn't bag of words supposed to be a method to generate features (one for each word)? How can BOW itself be a feature? Or does this simply mean we have a feature for each word as in BOW, besides all the other features mentioned?
  • how can a gazetteer be a feature?
  • how can POS tags exactly be used as features ? Don't we have a POS tag for each word? Isn't each object/instance a "text"?
  • what is global document information?
  • what is the feature trigger words?

I think all I need here is to just to look at an example table with each of these features as columns and see their values to understand how they really work, but so far I've failed to find an easy to read dataset.

Could someone please clarify or point me to some explanation or example of these features being used?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

isn't bag of words supposed to be a method to generate features (one for each word)? How can BOW itself be a feature? Or does this simply mean we have a feature for each word as in BOW, besides all the other features mentioned?

We have a feature for each word as in BOW. Yes, that's many features.

how can a gazetteer be a feature?

gazetteer = a list of words. The gazetteer feature value is 1 if the word is in the gazetteer, 0 otherwise.

how can POS tags exactly be used as features ? Don't we have a POS tag for each word? Isn't each object/instance a "text"?

we have a POS tag for each word; features are computed for each word.

what is global document information?

These are features computed on the entire document, e.g. length, topic, etc.

what is the feature trigger words?

From {1}: "A trigger word indicates that the tokens surrounding it are' probably a proper name and may reliably pernfit the class or even subclass 2 of tile proper nmne to be determined. For example, 'Wing and l'rwer Airlines' is ahnost certainly a company, given tile presence of the word 'Airlines'. "

References:

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.