1
$\begingroup$

I have a time series $S_i$ that I want to train a regressor on to predict the next point in the time series. I want to split the data into training and validation sets, and also scale the data in the range $[0, 1]$. The order of processing + training I have is the following

  1. Split $S_i$ into training and validation sets. The first 80% of the data is training, while the last 15% of the data is validation. The 5% left out acts as a "buffer" to ensure the two data sets are not too "close" to each other, chronologically speaking. This should help mitigate information leaking from the validation set into the training set
  2. Learn a MinMax scaler on the training data only, and apply this scaler to the training data and validation data
  3. Train a model on the training data, validate on the validation data

The issue with the process above, is that even though we only learn a scaler on the training data in step 2), I feel there is some information leakage within the training data; if the model is set up so that it iterates through every point $S_i$ in the scaled training data and attempts to predict the next point $S_{i+1}$, then some information from the scaler (which has used all data points in the training data) has leaked into the entire training data set.

One way that I have thought to get around this is to re-learn a MinMax scaler at every iteration of training using only the past data points that the model has already "seen". But this seems inefficient. Is there a better way to prevent information leakage when scaling? Is it necessary to prevent information leakage of this kind?

$\endgroup$

2 Answers 2

1
$\begingroup$

You can use your training set in anyway you like. Data leakage between the training and validation (or test) is important. Stepwise min/max scaling would create min/max values for each time step, which is inefficient (and unnecessary I believe) as you've also commented. Similarly, you can't apply transformations such as Box-Cox, STL decomposition etc. which is also common in time series analysis, easily.

$\endgroup$
3
  • $\begingroup$ Thanks, I'll take your word for it. Do you have any links / references / articles that say the same? $\endgroup$
    – PyRsquared
    Sep 30, 2019 at 10:12
  • 1
    $\begingroup$ @PyRsquared a simple tutorial following a similar idea: machinelearningmastery.com/… $\endgroup$
    – gunes
    Sep 30, 2019 at 14:11
  • $\begingroup$ appreciate it. I follow that site, I dont know how I missed that one. Thanks again! $\endgroup$
    – PyRsquared
    Sep 30, 2019 at 15:09
0
$\begingroup$

No... for training data for time-series data you cannot do whatever you like. The original reasoning is correct, you are introducing leakage into your data if you are not scaling with only past data. This makes sense: imagine a model is in production and you need to feed it data to get a prediction, say for something predicting the stock market. You performed scaling during training, so consistent with that you must scale the data now in production. You can only scale with past data at this point because that's all you have available (the future hasn't happened yet), and so to be consistent this is what you have to do during training.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.