I have a conceptual question regarding the use of the rolling window approach for training and validating a recurrent neural network (LSTM or GRU) on time series data.

I have daily time series data from 2013 to 2018 (~1800 time steps). So far I used the rolling window approach only for training the model. In particular, I created a fixed split for training and validation sets. E.g. first 1-1300 observations for training and 1301-1800 for validating. Within the 1300 training observations I created 1270 batches with a window size of 30. Hence:

1.    train_batch_x: 1-30,      train_batch_y: 31
2.    train_batch_x: 2-31,      train_batch_y: 32
1270. train_batch_x: 1270-1299, train_batch_y: 1300

The same structure was applied on the validation data. The 500 validation samples were converted to 470 batches of window size 30.

Thus, training and validation input data was transformed in 3D arrays with dim(batch size, window size, features) leading to the following dimensions:

dim(train_x) = (1270, 30, 14)
dim(train_y) = (1270)
dim(valid_x) = (470, 30, 14)
dim(valid_y) = (470)

Since I am not happy with an arbitrary split after 1300 observations I searched for better options and came across walk-forward validation. Is it applicable in my use case? Also, after reading about it in different sources I am quite confused whether I implemented it already in the approach described above or not. After all I only trained 1 model using the rolling window approach. But it seems that in the walk-forward validation many models will be trained.

Lastly, would it be an option to split my data multiple times? E.g.

1. train: 2013-2014, valid: 2015
2. train: 2014-2015, valid: 2016
4. train: 2016-2017, valid: 2018

Hence, I would train 4 different models using the rolling window approach initially described. However, which model should I then take for evaluating new data? the one with the highest validation accuracy? Or is the whole idea of the different models stupid?

I hope my question is clear. I would be very happy if you give me feedback on my thoughts and approach and give me some advice on how I should do the splitting of training and validation data best.

Thanks a lot in advance!

  • 1
    $\begingroup$ Did you find any solution? $\endgroup$ – Jemshit Iskenderov Oct 20 '18 at 18:51
  • $\begingroup$ Not really. In the end I used expanding window validation with 5 folds for training and validation and 1 fold for testing. Meaning: 1. Train on 1 and validate on 2 fold. 2. Train on 1,2 and validate on 3 fold. 3. Train on 1,2,3 and validate on 4 fold. 4. Train on 1,2,3,4 and validate on 5 fold. Test on 6th fold. $\endgroup$ – hokage555 Apr 13 '19 at 14:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.