Relying on the documentation provided by scikit-learn (https://scikit-learn.org/stable/modules/cross_validation.html#cross-validation-of-time-series-data), the TimeSeriesSplit method is implemented such that each training set is a superset of the previous ones. Can anyone provide theoretical justifications behind this choice ? Why not using training set of constant size (while still being careful about the time dependency of course) ?
First of all, you can set a constant training size via
max_train_size parameter, check the description page of TimeSeriesSplit. Depending on your problem, you might very well think that using all the data available to you is not necessary (e.g. especially if your data have a large time span), or having CV is computationally cumbersome. If your data is scarce, you might consider using all you have. The rationale of using training folds that are super sets of the previous ones is not throwing away any possibly useful data. However, the decision to apply the CV scheme changes with respect to problem domain and the situation of data.