Reinforcement Learning & Text Mining

Question

I was wondering if one could use Reinforcement Learning (as it is going to be more and more trendy with the DeepMind & AlphaGo's stuff) to parse and extract information from text.

For example, could it be a competitive approach to structured prediction such as

• Named Entity Recognition (NER), i.e. the task of labelling New York by "city", and New York Times by "organization"
• Part-of-speech tagging (POS), i.e. classifying words as determinant, noun, etc.
• information extraction, i.e. finding and labelling some target information in texts, for insance 12/03 is date given the context meaning 3 December and has the label "expiry date"

What would be a relevant modelling to do these tasks?

Rather naively I would think of a pointer that read the text from start to end and annotate each 'letter' by a label. Maybe it would learn that neighbouring letters in a 'word' share the same label, etc. Would it be able to learn long-term dependencies with this approach?

I am interested by any ideas or references related to this subject.

Answer 1

In these tasks full true output is known during the training; it makes the task somewhat easier than in general Reinforcement Learning setup. A setup when a good policy for training data is already known, and the task is to find a good policy which works on unseen data is sometimes called 'Imitation Learning'.

You may find papers about SEARN, DAgger and LOLS interesting; they explore these ideas. Start with http://arxiv.org/abs/1502.02206 and explore referenced papers. https://github.com/JohnLangford/vowpal_wabbit uses these techniques for structured prediction.