2
$\begingroup$

Reading through the scikit learn documentation on time series cross validation (https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation-of-time-series-data):

Time series data is characterised by the correlation between observations that are near in time (autocorrelation). However, classical cross-validation techniques such as KFold and ShuffleSplit assume the samples are independent and identically distributed, and would result in unreasonable correlation between training and testing instances (yielding poor estimates of generalisation error) on time series data. Therefore, it is very important to evaluate our model for time series data on the “future” observations least like those that are used to train the model. To achieve this, one solution is provided by TimeSeriesSplit.

Informally, that makes sense to me. However, is anyone aware of a formal proof (i.e., book or journal article) that shows the statement is correct?

$\endgroup$
1
$\begingroup$

On the contrary, cross-validation for time series does make sense when the model involves an autoregressive structure with uncorrelated errors. For a formal proof, see Bergmeir, Hyndman & Koo (CSDA 2018): https://robjhyndman.com/publications/cv-time-series/

$\endgroup$
  • $\begingroup$ Hi Rob, thanks for your reply. I noticed that you're the author of this paper, it's always good to have an expert on the subject. My data is both autoregressive and nonstationary. The problem I'm facing is early stopping (for a neural network) which is a similar problem to cross validation. $\endgroup$ – swmfg Feb 18 at 11:23
  • 1
    $\begingroup$ Rob a link is not a high quality answer here (even if it is a really good link :). Please expand your answer with a line of reasoning. $\endgroup$ – Alexis Mar 29 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.