7
$\begingroup$

When implementing some NLP project, such as text segmentation, Name Entity Recognition, does using trigram guarantee to perform more accurately than bigram?

$$ Trigram: p(s_t\mid s_{t-2}, s_{t-1}) $$ $$ Bigram: p(s_t\mid s_{t-1}) $$

EDIT: I was using an HMM to do NER on citation records(publications). I was using the bigram in my implementation. The accuracy was ok. I see Michael Collins' NLP class on Coursera where he uses a trigram HMM to do POS tagging. So I was wondering if trigram will boost the performance significantly or just a little bit. And I'm also curious if in any case trigram will perform worse than bigram.

whuber has already given a very good overview of the advantages and disadvantages of trigram in the comments.

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Some context to this question or further explanation would help, because on the face of it the answer is obvious: because trigrams include all the information in the bigrams, then any reasonable use of the trigrams cannot possibly be worse. Having said that, it's not hard to imagine circumstances where trigrams appear to perform worse: one might over-fit a model with them, for instance. But that depends on the application, the context, and the skills of the applier, among other things. $\endgroup$
    – whuber ♦
    Commented Aug 6, 2013 at 19:43
  • $\begingroup$ Thanks whuber, I've edited the questions. I think your comment has been already intuitive. Thanks a lot $\endgroup$
    – xiaoyao
    Commented Aug 6, 2013 at 20:27

2 Answers 2

Reset to default
1
$\begingroup$

As whuber explained in his comment, it depends on many factors, the most important one I believe being the information that the train set contains. E.g. if the train set is small, you're likely to have unseen trigrams, which will cause issues when tagging the test set. The choice of the size of n-gram can be seen as a bias–variance compromise .

Improve this answer
$\endgroup$
0
$\begingroup$

In bigram we consider past one words and in trigram we consider past two words. It can be happened that past two words itself happen less time and when it happens it contains all those probable words in same, more or less frequency. In my train set, trigram probability were same for two words where bigram probability were different with a great difference. So, it depends on the train set and test set which model will give the best answer.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.