I created a predicative model using neural networks and applied in on a time series. This is how I split my data:

train: 1, 2, 3, 4, 5, 6 -> 7, 8
test:  3, 4, 5, 6, 7, 8 -> 9, 10

The answer Using k-fold cross-validation for time-series model selection provides a similar solution to mine although I skip the initial part of the time series in the test data.

However, according to an answer in another question:

Leakage is when your test data is, in some form, also part of your train data.

If the k-fold validation used data from the training set in the test set how does it not induce data leakage?

  • 2
    $\begingroup$ Why did you split your test data? Why does your test data include training data? k-fold would be done on the train data, testing would then be done on a whole other separate set, there is no more k-fold on the test data. $\endgroup$ Sep 30, 2020 at 9:58
  • $\begingroup$ @user2974951 I split the test data so I can check the model's performance on it. It's the input and output for the model. As for the train data being present in the test data, the time series is not big (200-300 days) so the model would not have much to train on. $\endgroup$
    – Marcus
    Sep 30, 2020 at 10:28
  • 1
    $\begingroup$ That's not how this works. First you split your data into train and test set (which are mutually exclusive), then you perform whatever you have to on your train set, such as k-fold CV, obtain your final model and then you test it once on your test set. There is no splitting of the test data, the test data is what it is. If you cannot afford to split your data into train and test set then you can do rolling CV for time series. See robjhyndman.com/hyndsight/crossvalidation $\endgroup$ Sep 30, 2020 at 10:46


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.