Can we randomly select training and test sets in time series?

Question

Suppose I have a time series dataset with $t=1$ to $t=100$, and I want to take 80% (80 observations) as a training set and 20% (20 observations) as a test set. This link says that we cannot randomly select the observations:

In the case of time series, the cross-validation is not trivial. We cannot choose random samples and assign them to either the test set or the train set because it makes no sense to use the values from the future to forecast values in the past. In simple word we want to avoid future-looking when we train our model. There is a temporal dependency between observations, and we must preserve that relation during testing.

To me, I do not think so (EDIT: in general). I do not think that we are using the future to predict the past. I mean, there is no prediction in the sense of time. In fact, after taking a training set, we calculate predicted values for the test set. From predicted values under different models (model of $y$ as a function of $t$), the test set "chooses the best model".

In summary, we do calculate predicted values, we do not make prediction (in the sense of time). Therefore, I do not think that there is a barrier preventing us from randomly selecting training and test sets.

What do you think?

EDIT: I think we can do a random selection if we model $y$ as function of $t$. We may not be able to do that if we are model $y$ as a function of previous $y$ and $t$.

Answer 1

You can't meaningfully do a random assignment to training and test samples, at least not if your model should be taken seriously.

By "taking a model seriously", I mean that it could be used in a production context. For time series data, this usually involves taking data up to a certain time point, and predicting future data. Obviously, in such a context, you can't on Friday use next Sunday's data to predict next Saturday's - because you haven't observed Sunday's data yet. But this is what you would be doing if you train your data with a random assignment.

Or put another way: how would you use your model trained on future data to actually predict tomorrow's data point? What would you use as predictors, if it calls for using next week's observations?

There are special cases where you can use "future" data, e.g., if you are interpolating in time. But even then, you should think about how exactly you assign data to the training and the test set, and a simple random assignment would likely lead you astray.

Answer 2

If your data is time-dependant, you can’t randomly split train and test in a time-series. The ideal would be to use a threshold and use it as a way to split.

E.x. From t=1 to t=80 in train and from t=81 to t=100 in test.

If you don’t do this, you will incur in data leakage. And yes, you are using the future to predict the past.

As a example, suposse in your train end up t=60 and in your test t= 40, you are training with the future and predicting the past.

If your data has a date field but your response variable it’s not time dependant, you can drop the date and do a random split.