Validity of basic train - test - split for a time series using a RNN

I am trying to determine if a simple train-test-split is valid for a time series if I use a Recurrent Neural Network (LSTM). Lets say I have samples (x) which consist of 2 days values (time steps) and the y variable represents the day after (ie. I have day 1 and 2 as one of the x samples and want to predict day 3 (y)). In this situation, do we need to make a train test split based on dates, or I can just do the basic split since the RNN will have as input always the 2 days prior the one I am trying to predict? I trained using the simple split and got better results than using date related split, so I am trying to check if I am not "lying" myself doing the wrong split. I know that common time series techniques need to split acording to dates but I am unsure if a RNN does to.

You should respect the time order while splitting the dataset. This is simply because you're also peeking into the future for other samples in the learning process.

To give a simple illustration, let's say you've the following training samples:

b,c,d --> e
c,d,e --> f


while trying to estimate f from c,d,e you also learn some relationship between c,d,e. And, the algorithm can use this knowledge embedded somewhere in the architecture to generate better guesses from b,c,d to e, since it already knows something between c,d,e.

• Acording to your example, if I had also on the train sample e,f,g->h (after sample 1 and 2, so train set is time ordered ) but I test the model on d,e,f->g, I will be "cheating" because I used a sample for the train that was ahead in time regarding the test sample I am evaluating ? On another side, what if I use features instead the time series to predict the y values, would it be ok the normal random split, unless those features were autocorrelated (those feature would be a time series also since I got them for each day)
– JmML
Jan 28, 2022 at 11:08
• That is right. You peek into future trends during training. Jan 28, 2022 at 11:10
• @JmML sorry, I didn't see your edited comment. For your second question, the principle is simple, if there is temporal dependency between the features, you need to respect that. Jan 28, 2022 at 21:53