Suppose I have a time series dataset with $t=1$ to $t=100$, and I want to take 80% (80 observations) as a training set and 20% (20 observations) as a test set. This link says that we cannot randomly select the observations:
In the case of time series, the cross-validation is not trivial. We cannot choose random samples and assign them to either the test set or the train set because it makes no sense to use the values from the future to forecast values in the past. In simple word we want to avoid future-looking when we train our model. There is a temporal dependency between observations, and we must preserve that relation during testing.
To me, I do not think so (EDIT: in general). I do not think that we are using the future to predict the past. I mean, there is no prediction in the sense of time. In fact, after taking a training set, we calculate predicted values for the test set. From predicted values under different models (model of $y$ as a function of $t$), the test set "chooses the best model".
In summary, we do calculate predicted values, we do not make prediction (in the sense of time). Therefore, I do not think that there is a barrier preventing us from randomly selecting training and test sets.
What do you think?
EDIT: I think we can do a random selection if we model $y$ as function of $t$. We may not be able to do that if we are model $y$ as a function of previous $y$ and $t$.