# How to train a RNN language model?

I want to train a RNN-based language model from https://arxiv.org/pdf/1409.2329.pdf for next word prediction. How to split the sentences from the dataset into input and ground truth during the training?

Let's say I have the following training sample: The quick brown fox jumps over the lazy dog.

Does it makes sense to take every possible separation of this sentence? Doesn't this lead to overfitting?

input="The"                                     GT="quick"
input="The quick"                               GT="brown"
input="The quick brown"                         GT="fox"
...


Or can I just use only one last word as ground truth since the P(dog | The quick brown fox jumps over the lazy) already calculate probabilities of all previous words?

input="The quick brown fox jumps over the lazy" GT="dog"


In other words, typical discriminative supervised learning tries to learn $$p(y|x)$$. On the other hand, language models learn $$p(x)$$. As you can see there is no need for any labeled "$$y$$"
The way this works with autoregressive models such as RNNs is that you decompose it as a bunch of conditional probabilities $$p(x) = \prod_j p(x_j|x_{.