0
$\begingroup$

I have a question that is quite simple, but I can't wrap my head around it.

Say I have hourly data for one year, say 2020, and I split this data into a train and test set where the training set is the first 11 months. I estimate the coefficients for these 11 months according to this model: $$ Y_t = \beta_0 + \beta_1 Y_{t-24} + \beta_2 Y_{t-168} + \varepsilon_t. $$ That is, $Y_t$ is predicted by using the value from the same hour 1 day ago, and the same hour 7 days ago.

Now, if I want to predict the first day and hour of the test set "2020-12-01 01:00" I would have to use the value from 1 and 7 days before, that is, "2020-11-30 01:00" and "2020-11-24 01:00".

My question is, can I do this, since these values are used to estimate the coefficients, or is the first value I can forecast "2020-12-07 00:01"?

$\endgroup$

1 Answer 1

1
$\begingroup$

You can use these values as regressors (right-hand-side variables) without worrying about leaking the training data into the test set. Your target values (left hand side) will all be unseen, and this is what matters.

However, by splitting your data only once you only get a single one-step-ahead forecast on the test set. All the other forecasts will either be multiple steps ahead or they will be based on other forecasts (by iterative substitution of the necessary but unavailable actual data with their forecasts) rather than actual data. Therefore, instead of splitting the data as you have intended you may want to use time series cross validation via rolling windows or expanding windows. See Hyndman & Athanasopoulos "Forecasting: Principles and Practice" section 3.4.

$\endgroup$
6
  • $\begingroup$ Okay, thank you for your answer. So, the only thing to worry about is that I don't use any of my test data when estimating the coefficients? So this would mean that as soon as I know the last value of my training set, say I know this at "2020-12-01 00:00" I could predict all hours for that day? Thanks for that tip, I am re-estimating my coefficients every day in my model, with both rolling and expanding windows. I will check that source out since I could use some more knowledge about this subject. $\endgroup$ Nov 19, 2022 at 13:55
  • $\begingroup$ @SimonRydstedt, I am not entirely sure what you mean in your comment. The problem with autoregressive time series is that what is the dependent variable one day becomes an explanatory variable the next day. This is why I suggest sticking to time series cross validation, not the vanilla cross validation. $\endgroup$ Nov 19, 2022 at 15:18
  • $\begingroup$ Could you expand what you mean by "vanilla" cross validation? Would a sliding window be a time series cross validation, or do you mean that I should use for example 10-folded cross validation where every possible train/test variety is tried and compared? $\endgroup$ Dec 2, 2022 at 13:18
  • $\begingroup$ @SimonRydstedt Vanilla cross validation (CV) such as leave-one-out CV of k-fold CV disregards the time order and is not suitable for many time series models. Time series cross validation in the form of rolling or expanding windows (see the link in my answer) takes the time order properly into account. Use the latter. $\endgroup$ Dec 2, 2022 at 14:37
  • $\begingroup$ Thank you for the answer. I read the link, really good information on many areas there. I will use rolling windows in my project after comparing both. Otherwise the CV where all possible combination of train/test look interesting. Isn't there a version of this suited for TS? $\endgroup$ Dec 2, 2022 at 20:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.