As this paper (http://www.statnlp.org/research/ta/rnn_truecase.pdf) illustrates, a char rnn can be used to true-case some text i..e capitalize the necessary characters.

The paper just says it uses a beam search after the softmax over characters at the final layer of the char rnn. How can you do that?

My problem is:

  1. If we want to generate the truecase text then, clearly rnn is not well suited as it will try to generalize rather than memorize the text. So, how can you expect the same output except in truecase. They use a beam search, but again it shouldn't work. Can't expect rnn to output the same sentence as the input. Is there some special training trick here?

  2. How do you train it? What about the first character of text, it has no context, so how can you use RNN to predict truecase character or even train it?

Thanks for the help.

  • $\begingroup$ 1. Instead of requiring the RNN to output "truecase" text, have it output just the case (upper/lower) of each letter. $\endgroup$ – Sergey Slepov Mar 27 '17 at 8:18
  • $\begingroup$ 2. Introduce a lag between input and output? $\endgroup$ – Sergey Slepov Mar 27 '17 at 8:18
  • $\begingroup$ @SergeySlepov solving sequence tagging char by char using an RNN... $\endgroup$ – Rafael Mar 27 '17 at 8:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.