Machine Learning for Time Series: Train and test set overlap due to lagged target as feature – problem of data leakage?

Question

Situation:

My objective is to apply Machine Learning (for regression problems). Therefore, I have a panel dataset of time series with daily fund data from 2018-01-01 to 2020-12-31. My target is fund flow. To apply ML models, I wanted to split my dataset into train and test set, using 2018-01-01 to 2019-12-31 for training and 2020-01-01 to 2020-12-31 for testing. Among others, I also use the lagged target as features (as well as other lagged variables of my time series) up to a lag of 1 year before the actual Date (t-365)

Question: To predict the fund flow at 2020-01-01 (first date of test data) I use lagged time series features such as lagged fund flow (t-365). Therefore, this flow data used as feature here actually is from 2019-01-01. This data is appears as target (and features) also in the training set. I'm wondering whether this is an issue similar to data leakage. As I understand, my test dataset uses the exact data as features, my train data has already seen. Therefore the model knows large parts of this data (the closer the target date to the train set the more of the data the train set has already seen). I'm wondering if this is an issue in predicting time series? If I use feature with a lag of 365 days, do I need to keep a gap of 365 days between train and test set?

Haven't found much in the literature so far, so your help is appreciated! Thanks for your support!

Answer 1

Yes, of course this is an issue. The most obvious case where this pops up is in autoregressive models, where we use lagged values of the time series itself as a predictor. It is the exact same problem as for any other predictor we don't know with certainty in the future, e.g., if we use weather data as a predictor.

This is commonly addressed by forecasting the predictor itself and feeding this forecast into the model. If there are interrelationships between the predictor and the target, one typically runs the forecasting system step by step into the future. In this method, it makes most sense to use an expectation forecast for the predictor (which is not necessarily what we are aiming for for the focal variable, depending on our accuracy measure, Kolassa, 2020).

A more sophisticated approach would be not to account for the uncertainty in the forecast of the predictor. For instance, one could re-run the prediction multiple times, each time feeding in a sample from the predictive distribution of the predictor. This will do a better job at capturing the uncertainty in the focal variable.