How to understand the label-bias problem in HMM? How can I understand the label-bias problem in Hidden Markov Models? And why is CRF able to solve this problem?
 A: Label-bias is not a problem for HMM,because input sequence is generated by the model. By global normalization, CRF model avoid this problem.
A: Based on Section 2 of "Conditional Random Fields:  Probabilistic Models for Segmenting and Labeling Sequence Data" by Lafferty, J. et al,
I think it is "states with a single outgoing transition effectively ignores their observation. More generally, states with low-entropy next state distributions will take little notice of observations".
Honestly, I'm not quite sure if this is the Label Bias Problem. Because, I don't know why this is a problem. Aren't the next state distributions inferred from the training data? So low entropy next state distributions happen because that is what the data has..then the problem isn't the model.. it's the data's...
HTH.   
A: Suppose a simple finite state machine which was developed for named entity recognition. 
In those kinds of machines, the states with a single outgoing transition effectively ignore their observation. In other words, the states with a single transition simply have to move to the next state without considering their current observation. More generally, states with low-entropy next state distributions will take little notice of observations.
Reference:
J. D. Lafferty, A. McCallum, and F. C. N. Pereira, “Conditional random fields: Probabilistic models for segmenting and labeling sequence data”, in Proceedings of the eighteenth international conference on machine learning, ser. ICML ’01, San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2001, pp. 282–289, ISBN: 1-55860-778-1. [Online]. Available: http://dl.acm.org/citation.cfm?id=645530.655813.
A: CRF is a solution for MEMM and NOT for HMM.
In markov model Label-bias is not a problem ,because input sequence is generated by the model (Farhana Liza). in MEMM, while calculating the transition probabilities, from every position (AKA state), the probabilities sums up to 1. 
So whats the problem? 
Lets say we have a state that is very UNlikely to happen, but when it does, with a very high probability (even 1) it would happen again. Now, if we have a long chain of states, there is a higher probability to stay in that position forever even though it is a state that is unlikely to happen!
In the CRF model we are using GLOBAL NORMALIZATION, which takes care of it and sums up all of the transition probabilities to 1.
Good luck!
