As this paper (http://www.statnlp.org/research/ta/rnn_truecase.pdf) illustrates, a char rnn can be used to true-case some text i..e capitalize the necessary characters.
The paper just says it uses a beam search after the softmax over characters at the final layer of the char rnn. How can you do that?
My problem is:
If we want to generate the truecase text then, clearly rnn is not well suited as it will try to generalize rather than memorize the text. So, how can you expect the same output except in truecase. They use a beam search, but again it shouldn't work. Can't expect rnn to output the same sentence as the input. Is there some special training trick here?
How do you train it? What about the first character of text, it has no context, so how can you use RNN to predict truecase character or even train it?
Thanks for the help.