BiLSTM-CRF is a common model for sequence tagging (POS tagging, NER, ect.). What are the advantages of combining BiLSTM and CRF? What is the role of each one of the parts in this combination?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
TL;DR: BiLSTM knows about the language, CRF knows the internal logic of the labeling.
With a plain BiLSTM followed by a classifier, each classification decision is conditionally independent. Linear-chain CRF explicitly models dependencies between the labels as a table with transition scores between all pairs of the labels.
If the labels follow strict internal syntax, this is extremely easy for the CRF to learn. For NER, there are several ways of encoding the output, but they typically encode at least: Begining, Iinside and Outside of an entity and obviously these can only be in a syntactically well-defined order. CRF will very quickly catch that it is impossible for instance that I-LOC would follow O, it must always follow B-LOC.
Another example, BiLSTM might be unsure if it should place B-PER on a position or one position later and end up outputting both of them because they are conditionally independent. CRF layer that knows that this is unlikely and enforces the internal logic of the tags and would output B-PER, I-PER.
$\begingroup$
$\endgroup$
Totally agree with Jindrich, I work in Chinese Grammatical Error Correction (CGEC) and BiLSTM-CRF is the fundamental structure of our network.
Basically, BiLSTM is used to take the context into consideration, while CRF is capable of considering longer relationships.
Although CGEC is different from NER, it shares the same logic. In CGEC we need to tag the grammatically incorrect characters with a label. BiLSTM reads the text embedding and provides a probability about NER encoding, CRF uses the probability as an input , find the shortest path using the Viterbi algorithm, and ultimately gives the final result.
The reason that CRF can learn the internal logic is that based on BiLSTM it gives the probability distribution and emission score matrix as the input of CRF, CRF would learn from the input to know how to label the sequence.
