Suppose I'm trying to predict time series with a neural network. The data set is created from a single column of temporal data, where the inputs of each pattern are [t-n, t-n+1, ... , t], t being the time step and n the embedding size, and [t+1] being the target (predicting the "next step" of the series). Here is the question: if I use such a data set for NN training, should I shuffle it? Shuffling is usually done to avoid overfitting by preventing the NN from memorizing the pattern order. However, in case of time series, could pattern order be a form of information that I shouldn't deprive the NN of? The issue becomes even more severe if recurrent NNs are concerned, where every pattern recurrently receives information about the previous pattern: would a RNN benefit if patterns were in order?
|
|
|||
|
|
|
Say you have a single time series you want to learn on. Then you can use the first half for the development of your model and the second half for testing. You now cut your two halfs into windows individually and can shuffle those of the training set. From what I know, the shuffling is actually not because of generalization, but because of optimization. It is sometimes more efficient to optimize a sum of functions (in this case, one loss function per time window) if you do not look at their sum, but estimate the gradient by looking at the gradients of a subset of the sum. Look at recent publications by Nicolas Le Roux and Marc Schmidt for this topic. |
|||||||||||
|
|
If you are using a not-recurrent NN like a traditional MLP you don't NEED to shuffle dataset especially if you're using a batch learning algorithm. Anyway, in my experience it's a good idea to shuffle datasets for a faster training and more clear results. I can suggest you "Efficient BackProp" by LeCun et al., where you can find a useful set of tips & tricks on the use of NNs. |
|||||||
|
