Validity of basic train - test - split for a time series using a RNN

Question

I am trying to determine if a simple train-test-split is valid for a time series if I use a Recurrent Neural Network (LSTM). Lets say I have samples (x) which consist of 2 days values (time steps) and the y variable represents the day after (ie. I have day 1 and 2 as one of the x samples and want to predict day 3 (y)). In this situation, do we need to make a train test split based on dates, or I can just do the basic split since the RNN will have as input always the 2 days prior the one I am trying to predict? I trained using the simple split and got better results than using date related split, so I am trying to check if I am not "lying" myself doing the wrong split. I know that common time series techniques need to split acording to dates but I am unsure if a RNN does to.

Thanks in advance

Answer 1

You should respect the time order while splitting the dataset. This is simply because you're also peeking into the future for other samples in the learning process.

To give a simple illustration, let's say you've the following training samples:

b,c,d --> e
c,d,e --> f

while trying to estimate f from c,d,e you also learn some relationship between c,d,e. And, the algorithm can use this knowledge embedded somewhere in the architecture to generate better guesses from b,c,d to e, since it already knows something between c,d,e.