4
$\begingroup$

In NER(Named Entity Recognition) example in python-crf package website we see this function as feature generator:

def word2features(sent, i):
word = sent[i][0]
postag = sent[i][1]
features = [
    'bias',
    'word.lower=' + word.lower(),
    'word[-3:]=' + word[-3:],
    'word[-2:]=' + word[-2:],
    'word.isupper=%s' % word.isupper(),
    'word.istitle=%s' % word.istitle(),
    'word.isdigit=%s' % word.isdigit(),
    'postag=' + postag,
    'postag[:2]=' + postag[:2],
]
if i > 0:
    word1 = sent[i-1][0]
    postag1 = sent[i-1][1]
    features.extend([
        '-1:word.lower=' + word1.lower(),
        '-1:word.istitle=%s' % word1.istitle(),
        '-1:word.isupper=%s' % word1.isupper(),
        '-1:postag=' + postag1,
        '-1:postag[:2]=' + postag1[:2],
    ])
else:
    features.append('BOS')

if i < len(sent)-1:
    word1 = sent[i+1][0]
    postag1 = sent[i+1][1]
    features.extend([
        '+1:word.lower=' + word1.lower(),
        '+1:word.istitle=%s' % word1.istitle(),
        '+1:word.isupper=%s' % word1.isupper(),
        '+1:postag=' + postag1,
        '+1:postag[:2]=' + postag1[:2],
    ])
else:
    features.append('EOS')

return features

As you see after appending meaningful features - like word.lower and ...- two features has appended.

features.append('EOS')

and

features.append('BOS')

My question is "What's meaning of BOS and EOS and what is the role of them?" You can see the completed tutorial there: python-crfsuite NER example

$\endgroup$
3

1 Answer 1

3
$\begingroup$

Based on the structure of the code, and my intuition, I would guess those stand for Beginning Of Sentence and End Of Sentence. Specifically, BOS is only appended when i == 0 (i.e., before anything else), and EOS is only appended when i == len(sent) (i.e., after everything else).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.