I was wondering if one could use Reinforcement Learning (as it is going to be more and more trendy with the DeepMind & AlphaGo's stuff) to parse and extract information from text.

For example, could it be a competitive approach to structured prediction such as

  • Named Entity Recognition (NER), i.e. the task of labelling New York by "city", and New York Times by "organization"
  • Part-of-speech tagging (POS), i.e. classifying words as determinant, noun, etc.
  • information extraction, i.e. finding and labelling some target information in texts, for insance 12/03 is date given the context meaning 3 December and has the label "expiry date"

What would be a relevant modelling to do these tasks?

Rather naively I would think of a pointer that read the text from start to end and annotate each 'letter' by a label. Maybe it would learn that neighbouring letters in a 'word' share the same label, etc. Would it be able to learn long-term dependencies with this approach?

I am interested by any ideas or references related to this subject.


1 Answer 1


In these tasks full true output is known during the training; it makes the task somewhat easier than in general Reinforcement Learning setup. A setup when a good policy for training data is already known, and the task is to find a good policy which works on unseen data is sometimes called 'Imitation Learning'.

You may find papers about SEARN, DAgger and LOLS interesting; they explore these ideas. Start with http://arxiv.org/abs/1502.02206 and explore referenced papers. https://github.com/JohnLangford/vowpal_wabbit uses these techniques for structured prediction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.