I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the training?
Let's say I have the following training sample:
The quick brown fox jumps over the lazy dog.
Does it makes sense to take every possible separation of this sentence? Doesn't this lead to overfitting?
input="The" GT="quick" input="The quick" GT="brown" input="The quick brown" GT="fox" ...
Or can I just use only one last word as ground truth since the P(dog | The quick brown fox jumps over the lazy) already calculate probabilities of all previous words?
input="The quick brown fox jumps over the lazy" GT="dog"