I am setting up a neural network that will predict the incoming customers at a store for the next seven days (the output is a list with seven numbers, one for each day). As input, I will give the neural network the last x weeks of data. The data contains the number of visitors for each day and information about that date, like what day it was (Monday, Tuesday, etc.). The optimal x has yet to be determined.

I split the data in a training- (80%) and testset (20%).

For training the neural network I am now using walk forward optimization where I take the first x weeks of the trainingset as input and use the next seven days after that as output. This is one input-output pair. For the next pair, I move the window one day. The input loses the first day, but gains the first day of the output of the previous pair. The output gains a whole new day. I hope the picture clarifies what I am talking about.

Walk forward optimization with a sliding window.

I wonder if this method will cause data-leakage, since I am giving the neural network output that it has already seen, one input-output pair before. I guess this would be solved by shifting the pairs by seven days instead of one, but this leaves me with very little data to train on. Keep in mind that I use this method for training the network. When testing the method, I am using data that the neural network has never seen before.

  • $\begingroup$ What you have written doesn't seem to align with the diagram provided. Could you clarify what is meant by "When testing the method, I am using data that the neural network has never seen before" as the diagram seems to show that the testing set in the top row is contained in the training set in the second row. $\endgroup$ – Ryan Apr 12 '19 at 14:55
  • $\begingroup$ Yeah the diagram doesn't represent what I am doing exactly, my bad. In the diagram it says "training set" while for me it should say "input for training". "Testing set" should be "output for training". Also, in my case I am not moving the window with the size of the "output for training"-set, but with one day. I hope this also clarifies what I mean with "When testing the method, I am using data that the neural network has never seen before". When the network is done training the way I just described, I test with the test-set (last 20% of the original data). $\endgroup$ – Marthijn Apr 13 '19 at 8:38

Your approach seems perfectly valid. Since you don't train your model on future data reserved for testing you avoid the problem of look-ahead bias.

You don't have to shift your training window forward by the seven days covered by your test period because even when you walk forward by only one day you are not using any future observations for training and constructing the forecast (apart from deterministic features such as day of week etc.).

A nice description of this procedure (also known as time series cross-validation) is given in the blog post Cross-validation for time series by Robert Hyndman.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.