I have recently learnt about the bootstrapping method and I am using it in my model tuning phase of my current project. I am working with time series data and therefore have decided to use a Stationary bootstrap which is a variation on the Moving Block bootstrap. I have decided to do this for two reasons, one so I can generate confidence intervals (uncertainties) of the models' accuracy to allow for more informed comparisons and secondly because I do not have many data points in each time series. By bootstrapping $B$ times I am able to just leave one forecasting horizons length out at the end of my train dataset, which maximises the amount of data points my model can train on.

I have however a relatively long forecast horizon and when using neural network models this results in relatively few samples to train on ~100-300.
I would like to augment the amount of samples that I have to measure whether or not adding more samples for the networks to train on will increase their measured performance.

It occurred to me that if I am using the bootstrap technique to compute standard statistics of my models I could also use it to increase the number of training samples.

My method would be to enhance the dataset by generating enough additional bootstrapped time series such that the number of possible samples is at a level at I want. I will then take each time series, that is, the original and the bootstrapped ones and from these generate samples which consist of $n$ lags which can be fed to the network like a supervised learning problem. As is common for time series forecasting problems. Is this valid?

I think it is. When bootstrapping, we are assuming the distribution of data points matches the true distribution and therefore if I can generate bootstrapped time series that have this distribution then these should also be valid 'sequences' my model could learn from. As I am feeding my network samples which contain just a windowed section of the time series in each sample, it also shouldn't matter that the samples come from the original time series or a bootstrapped one; if we assume that the data generating function of both the series is the same. Which I think should hold if the empirical distributions are agreed to be good approximators of the true distribution.

Similar techniques are used in computer vision and classification to make models more robust (adding noise to images etc.) and so I believe that a second result of increasing the number of samples could be that the model becomes more robust.

I googled a bit and I found this paper from Slawek Syml (recent winner of the M4 forecasting competition) and Karthik Kuber where they do propose data augmentation techniques for time series forecasting and under the further work section state bootstrapping could be investigated. I haven't found however anything that discusses the results of applying the technique nor have I found much on whether or not it has been used. Is there a reason for this?

This paper from Bergmeir, Hyndmann & Benitez use bootstrapping to create a new series that the models are trained on but as I understood it do not use it to 'extend' a time series with which more samples could be drawn from. Which would be my use case.

I found this question which is somewhat relevant but does not as far as I can tell deal with the bootstrapping case or answer my question directly.

I also asked a question here about time series data augmentation where I thought to add noise and or smoothing of the initial time series to increase the number of samples available to me. As of yet is unanswered and I now think that this bootstrapping idea would be a better solution to solve of my problem of time series data augmentation for short time series.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.