So I've been trying to work out exactly what temporal leakage is for a while now and I'm getting nowhere.

I'm not necessarily looking to code or anything, I'm more so interested in what it actually is and how it occurs, so if possible, please keep it as simple as possible.

Thanks for your time!

  • $\begingroup$ Could you describe an example of what you mean by "temporal leakage"? Where did you hear about it? From the name alone, I'm not sure exactly what you're referring to, though I'll take a stab at it. $\endgroup$
    – alltom
    Jul 18 '20 at 4:31

(Making a guess at what you mean by "temporal leakage"…)

When working with time series data, like the price of a stock over time, many of the ways you would typically split the data into training and validation sets lead you to believe that your model generalizes better than it actually does.

For example, if you have 1,000 samples of the price of MSFT over the course of a day, you might think you can select a random 500 samples to compose your training set and use the rest for validation. If you do that, your model will be surprisingly good, as if it has a miraculous ability to predict the future of Microsoft's stock price… That's because it has seen the future of Microsoft's stock price: the training set contains data before and after almost every point in the validation set. From the point of view of the validation data set, information from the future has leaked into your model.

The problem with k-fold validation is analogous.

To prevent information about the future from leaking into your model validation, ensure that all the data in your validation set come after all the data in your training set temporally.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.