I use random forest for time series forecasting.I have some features:

  1. lags.
  2. day of year,day of week,hours,minutes.

rf = RandomForestRegressor(n_estimators=1000,n_jobs=-1,bootstrap=False).fit(X_train, y_train)

rf-function use for learning random forest.X_train-features,y_train-values of time series,n_estimators-number of trees,(bootstrap=True)=use bootstrap,(bootstrap=False)=not use bootstrap

example of X_train:enter image description here

Should i write bootstrap=False in this case? If not, why?

  • 2
    $\begingroup$ For those of us who do not use Python, could you explain in words what the function is doing? $\endgroup$ Dec 14 '18 at 13:55
  • $\begingroup$ rf-function for learning random forest.X_train-features,y_train-values of time series,n_estimators-number of trees,(bootstrap=True)=use bootstrap,(bootstrap=False)=not use bootstrap $\endgroup$ Dec 14 '18 at 14:01
  • $\begingroup$ Thank you. What for and how is this bootstrap used, if specified to TRUE? You may edit the post to include new information; this way new readers will not have to dig through comments. $\endgroup$ Dec 14 '18 at 14:09
  • $\begingroup$ I have a poor knowledge of the theory since I am a new to this topic.as i understand it, bootstrap is used to normalize the distribution.In a random way,n samples are formed,in which the number of objects=the number of objects in the original sample.After that, trees are built on these samples,and final forecast is the average for trees. $\endgroup$ Dec 14 '18 at 14:21
  • $\begingroup$ I dont what happens inside the function when bootstrap = True. $\endgroup$ Dec 14 '18 at 14:27

Generally when modeling time series, you want to use walk forward cross validation, but lets first look at bootstrapping in the context of RF.

Bootstrap sampling in RF is an inbuilt sampling mechanism where at each tree, a new sample set is chosen. An important feature of bootstrap sampling is that repeated values are allowed. Hence, you could in theory have trees that are constructed with many duplicated values. The main benefit to bootstrapping is after you are done with the process you are left with samples not drawn. These are so called "out of the bag" samples, OOB for short. Since the model has never seen this data it can be used to measure model robustness similarly to a testing split. From an academic perspective, this allows you to use the entire dataset to train the model and not have to worry about splitting the data.

Now back to your original question. I would not use bootstrapping to model a time-series. Use a cross validate sampling scheme. As Richard Hardy mentions, using a random sampling technique distorts the structure of a time series. In addition, bootstrapping can cause too many duplicated values. Poor data representation is also an issue because due to the random nature you could end up over-sampling and under-sampling certain periods in the time series. Using a cross validated approach you get a more realistic result because you are simulating producing forecasts. Simply, you split the data into training and testing $ k $ times, with each $ k_i $ adding the first observation in the test set to the training set. Train and test the model $ k $ times. To get model performance, you average the testing error across the $ k $ iterations. This simulates walking forward a forecast; this is the way forecast models are implemented out of development.

One last thing to mention. If you are getting different results each time you trained the model with bootstrap = T it is likely because you have not set a seed. Set a seed so that random number generation is reproducible. This means the samples generation will be the sample each iteration. This will allow you to reproduce your results. Then compare the bootstrap = T with the bootstrap = F.

If you are looking for more on time series cross validation, here is a great explanation: https://robjhyndman.com/hyndsight/tscv/

Here is a link to documentation of the function you are using, might be helpful: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.