In order to create a training set for RNNs one typically takes a sequence and turns it into a training set using the sliding window approach. For example if the sequence is:
1 2 3 4 5 6 7 8 9
One can form a training set using a window size of 4 as follows:
1 2 3 4 2 3 4 5 3 4 5 6 4 5 6 7 5 6 7 8 6 7 8 9
In the above training set the first 3 columns are input features and the last column is the output (in a more realistic setting larger window sizes can be used).
Now actually this is a classical supervised machine learning training set. So I can use any supervised regression model such as linear regression or multilayer NNs. I can use an RNN model too. My question is this: what properties of the data sequence make RNNs a better predictor than linear regression or multilayer NNs? I anticipate some kind of answer like: when the past data is important for prediction. However, the past data (i.e., input features) is available for all methods. There must be some kind of structure in the inputs that makes RNNs suitable for prediction. If you can also also give some simple example sequences for illustration, it would be great.