Lagged variables, data leakage and machine learning

Question

I am reading a paper that fits a random forest (RF) to some data that is grouped by company and quarter. In the data engineering stage, the authors include 'lagged' variables of many of the explanatory variables to feed the RF.

My question: By lagging the variables, aren't we inducing a 'data leakage' problem? (They computed the lags before splitting into train/test) For instance, for a company-quarter combination that is in the TRAINING set and already has the info for a different_quarter-same_company combination that is in the TEST set, could we potentially lead to overly optimistic results because of the 'leakage'?

Thank you for your thoughts!

Answer 1

The answer is yes, this can pose data-leakage problems. However, this does not mean that it is wrong to use the lagged variable, but only that the data will have a time dependency that needs to be taken into account in the training process.

I suggest this article that explains very well the solution: https://medium.com/@soumyachess1496/cross-validation-in-time-series-566ae4981ce4