Why discriminative models are preferred to generative models for sequence labeling tasks?

Question

I understand that discriminative models, such as CRF(Conditional Random Fields), model conditional probabilities $P(y|x)$, while generative models, such as HMM(Hidden Markov Model), model joint probabilities $P(y,x)$.

Take CRF and HMM for example. I know that CRF can have a larger range of possible features. Apart from that, what else makes CRF (discriminative models) preferable to HMM(generative models) in sequence labeling tasks such as Part-of-Speech tagging and NER(Name Entity Recognition)?

Edit:
I found out that HMMs will have to model $P(x)$, while CRFs don't. Why would it make a big difference in sequence labeling tasks?

Answer 1

I think you pretty much nailed it in your Edit. Generative model makes more restrictive assumption about the distribution of $x$.

From Minka

"Unlike traditional generative random fields, CRFs only model the conditional distribution $p(t|x)$ and do not explicitly model the marginal $p(x)$. Note that the labels ${ti }$ are globally conditioned on the whole observation $x$ in CRFs. Thus, we do not assume that the observed data $x$ are conditionally independent as in a generative random field."

Answer 2

CRFs and HMMs are not necessarily exclusive model formulations. In the formulation you have above, X in the HMM is usually a state variable that is unobserved, so a generative model is somewhat necessary. In the CRF though, X is some feature vector that is observed and affects Y in the traditional way. But you can have a combination of both: a sequence of states and outputs where the state is unobserved, and a set of observed features that affects the conditional probabilities of the outputs given the states (or transition probabilities between states).

I believe that ultimately the CRF admits some more flexible models where the conditional probabilities are more dynamic, and could be affected by, for example, the output from several observations ago, or something like that. They can get awfully large and difficult to train when they start including many more free parameters like that though.