How to define train and test sets in financial time series for estimating machine learning parameters After reading some material, I found few options for defining train and test sets:


*

*Just splitting with no change.

*Accumulating/moving window of train set.

*Leave a relatively small (warming) period between test and train sets, and then use window again (including the warming period).


What should be the most accurate way for applying machine learning algorithms and parameters estimation?
 A: If you're still looking for insight regarding financial time series & machine learning, you might want to check out this article from the Journal of Economic Perspectives, which gives a great overview of various ML methods pertaining to Economics/Finance.
Essentially, the main problem you have is that most traditional machine learning techniques deal with cross-sectional data "where independently distributed data is a plausible assumption" (quoted from said article). However, since with Financial Time Series you, by and large, can't make that assumption, you're better off taking a totally different approach than the 'ole Training/Test set split-'em-up. Your best bet--as mentioned in that article (Seriously, it's really good)--may be to read up on Bayesian Structural Time Series (BFTS) (briefly mentioned in that article that you should be reading by now and described in more detail here and, well, I don't have the reps for a third link...).
Now, if you're just looking to do some run-of-the-mill Time Series estimation you can settle for the choose-the-model-with-the-lowest-out-of-sample-RMSE approach. However, that may cause you to forfeit all your "Machine Learning" name-dropping privileges. Just a warning...
Good luck!
A: Generally cross-validation is one of the methods to evaluate a model by splitting data into train and test data sets. Leave-one-out cross-validation splits the dataset, say n datapoints as (n-1) for train data and test on nth datapoint. this process is repeated until each data point serves as a test datapoint. This ensures fairness in splitting the training data and rigorous evaluation of the model.
