data leakage when scaling time series

Question

Suppose I want to forecast future values of $y$ past values of features $x$. In this example I am using:

• the training set goes from $t_0$ to $t_{15}$
• values from $x_{t_0}$ to $x_{t_{10}}$ to forecast $y_{t_{11}}$
• values from $x_{t_1}$ to $x_{t_{11}}$ to forecast $y_{t_{12}}$
• and so on until I use $x_{t_6}$ to $x_{t_{15}}$ to forecast $y_{t_{16}}$

I scale the my feature $x$ using only data in the training set (up to $t_{15}$)

Nevertheless when I try to predict $y_{t_{17}}$ you can see from the picture below that I use some data points that have also been used for scaling.

Is this leakage?

Answer 1

Formally speaking, for the prediction of $y_{t_{16}}$ and onwards, there is no leakage because the standardisation does not include information that would be unavailable at the time of prediction.

That said, the training does have some leakage in itself though as for example when we predict $t_{10}$ we use info from parts of the training set that occur from $t_9$ onwards. For any particular prediction at $t_i$, using information based on $t_{i+1}$ should be strictly avoided. To fix the training leakage, it makes sense, during training, to standardise the sample only with data that have been seen up until the point of prediction.

Rob Hyndman has a pretty authoritative blog post on the matter of cross-validation for time series. For something slightly more formal, Cerqueira et al. (2019) Evaluating time series forecasting models offers an excellent recent overview of many approaches (cross-validation in a blocked forms, out-of-sample methods, etc.) As more general reference for cases where the assumption of independence is compromised or where model extrapolation is likely it worth looking at Roberts et al. (2016) Cross‐validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure: it views the issue from an Ecology lens but it really detailed and accessible.